[jira] [Resolved] (PROTON-781) Implement the Reactive APIs in Ruby
[ https://issues.apache.org/jira/browse/PROTON-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-781. - Resolution: Fixed Fix Version/s: 0.10 > Implement the Reactive APIs in Ruby > --- > > Key: PROTON-781 > URL: https://issues.apache.org/jira/browse/PROTON-781 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-914) SSL.peer_hostname does not return the proper value.
[ https://issues.apache.org/jira/browse/PROTON-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-914. - Resolution: Fixed Fix Version/s: 0.10 > SSL.peer_hostname does not return the proper value. > --- > > Key: PROTON-914 > URL: https://issues.apache.org/jira/browse/PROTON-914 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9.1 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.10 > > Attachments: > 0001-PROTON-914-Fix-for-getting-the-SSL-peer-hostname-in-.patch > > > The method is returning the result code for the underlying C api rather than > the actual value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-914) SSL.peer_hostname does not return the proper value.
[ https://issues.apache.org/jira/browse/PROTON-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-914: Attachment: 0001-PROTON-914-Fix-for-getting-the-SSL-peer-hostname-in-.patch > SSL.peer_hostname does not return the proper value. > --- > > Key: PROTON-914 > URL: https://issues.apache.org/jira/browse/PROTON-914 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9.1 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Attachments: > 0001-PROTON-914-Fix-for-getting-the-SSL-peer-hostname-in-.patch > > > The method is returning the result code for the underlying C api rather than > the actual value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-914) SSL.peer_hostname does not return the proper value.
[ https://issues.apache.org/jira/browse/PROTON-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-914: Attachment: (was: 0001-PROTON-914-Fix-for-getting-the-SSL-peer-hostname-in-.patch) > SSL.peer_hostname does not return the proper value. > --- > > Key: PROTON-914 > URL: https://issues.apache.org/jira/browse/PROTON-914 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9.1 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > > The method is returning the result code for the underlying C api rather than > the actual value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-914) SSL.peer_hostname does not return the proper value.
[ https://issues.apache.org/jira/browse/PROTON-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-914: Attachment: 0001-PROTON-914-Fix-for-getting-the-SSL-peer-hostname-in-.patch > SSL.peer_hostname does not return the proper value. > --- > > Key: PROTON-914 > URL: https://issues.apache.org/jira/browse/PROTON-914 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9.1 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Attachments: > 0001-PROTON-914-Fix-for-getting-the-SSL-peer-hostname-in-.patch > > > The method is returning the result code for the underlying C api rather than > the actual value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-914) SSL.peer_hostname does not return the proper value.
Darryl L. Pierce created PROTON-914: --- Summary: SSL.peer_hostname does not return the proper value. Key: PROTON-914 URL: https://issues.apache.org/jira/browse/PROTON-914 Project: Qpid Proton Issue Type: Bug Components: ruby-binding Affects Versions: 0.9.1 Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce The method is returning the result code for the underlying C api rather than the actual API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-914) SSL.peer_hostname does not return the proper value.
[ https://issues.apache.org/jira/browse/PROTON-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-914: Description: The method is returning the result code for the underlying C api rather than the actual value. (was: The method is returning the result code for the underlying C api rather than the actual API.) > SSL.peer_hostname does not return the proper value. > --- > > Key: PROTON-914 > URL: https://issues.apache.org/jira/browse/PROTON-914 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9.1 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > > The method is returning the result code for the underlying C api rather than > the actual value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [GitHub] qpid-proton pull request: Proton 781 ruby reactor apis
On Mon, Jun 08, 2015 at 07:10:34PM +, mcpierce wrote: > GitHub user mcpierce opened a pull request: > > https://github.com/apache/qpid-proton/pull/36 > > Proton 781 ruby reactor apis > > The Ruby Reactor APIs plus example applications. Hi, all. If nobody objects, I'm going to merge this into trunk tomorrow (18 June) at COB EST unless anybody objects. This shouldn't have any impact on code outside of the Ruby bindings. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpju48VFZdNP.pgp Description: PGP signature
Re: Strange behaviour for pn_messenger_send on CentOS 6
On Wed, Jun 10, 2015 at 02:17:23PM +0100, Frank Quinn wrote: > You can recreate this on a CentOS 6.6 box by installing qpid-proton-c-devel > using yum from EPEL, then compiling the example application that comes with > it in /usr/share/proton/examples/messenger/send.c. > > There is no broker with these example applications - it's point to point. > > There is a matching recv.c application there too. If you start recv first, > it works fine, but if you don't, the send application hangs which I believe > is new behaviour. > > Again, this does not happen on my Fedora laptop - only on CentOS 6. I'm able to recreate this on RHEL6 as well using the RPMs from EPEL. I see the connection aborted notice in the trace and then it fails to exit. It appears to be repeatedly calling pn_transport_closed(). The stack trace I see on EL6 is: (gdb) backtrace #0 pn_transport_closed (transport=0x804fa28) at /home/mcpierce/Programming/Proton/proton-c/src/transport/transport.c:2802 #1 0x00140ea8 in pni_connection_capacity (ctx=0x804f920) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:166 #2 pni_connection_update (ctx=0x804f920) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:196 #3 pni_conn_modified (ctx=0x804f920) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:225 #4 0x00141071 in pn_messenger_process_transport (messenger=0x804cb40, event=0x805c520) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1201 #5 0x00141134 in pn_messenger_process_events (messenger=0x804cb40) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1252 #6 0x00141f83 in pni_connection_readable (sel=0x804f958) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:261 #7 0x00143425 in pn_selectable_readable (selectable=0x804f958) at /home/mcpierce/Programming/Proton/proton-c/src/selectable.c:204 #8 0x00141483 in pn_messenger_process (messenger=0x804cb40) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1310 #9 0x001415c8 in pn_messenger_tsync (messenger=0x804cb40, predicate=0x13da40 , timeout=-1) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1379 #10 0x00141a97 in pn_messenger_sync (messenger=0x804cb40, predicate=0x13da40 ) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1410 #11 0x00141c8c in pn_messenger_send (messenger=0x804cb40, n=-1) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:2119 #12 0x08048e59 in main (argc=-72537468, argv=0xb7ffe000) at /home/mcpierce/Programming/Proton/examples/c/messenger/send.c:102 The same backtrace on F22 is: (gdb) backtrace #0 pn_transport_closed (transport=transport@entry=0x60ac40) at /home/mcpierce/Programming/Proton/proton-c/src/transport/transport.c:2801 #1 0x77bbf0a8 in pni_connection_capacity (sel=0x60aaf0) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:166 #2 pni_connection_update (sel=0x60aaf0) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:196 #3 pni_conn_modified (ctx=0x60aa80) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:225 #4 0x77bbf135 in pn_messenger_process_transport (messenger=messenger@entry=0x605970, event=event@entry=0x61a240) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1201 #5 0x77bbf27b in pn_messenger_process_events (messenger=messenger@entry=0x605970) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1252 #6 0x77bbf6d9 in pni_connection_readable (sel=0x60aaf0) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:261 #7 0x77bbf7e8 in pn_messenger_process (messenger=messenger@entry=0x605970) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1310 #8 0x77bbf94f in pn_messenger_tsync (messenger=0x605970, predicate=0x77bbc420 , timeout=-1) at /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1379 #9 0x00401158 in main (argc=, argv=) at /home/mcpierce/Programming/Proton/examples/c/messenger/send.c:102 There's definitely a difference in the runtime behavior of Fedora vs. RHEL in this case. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpS6Fao6o8uN.pgp Description: PGP signature
Re: Strange behaviour for pn_messenger_send on CentOS 6
On Tue, Jun 09, 2015 at 10:41:51PM +0100, Frank Quinn wrote: > Just tried this and can recreate on a 64 bit laptop running CentOS 6.6 > natively too. The send application never exits if a receiver is not yet > ready. Can anyone else see this or am I going mad? Are these bits you've built or did you install them from RPM (not sure if you said before, so please remind me)? And are you running a Qpid or other broker? I ask because it seems you're getting a connection (no "connection refused" error) but something else is causing the failure, the SASL header mismatch. How is your SASL configuration setup on the broker to which you're connecting? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpjlOinpZ7_p.pgp Description: PGP signature
Re: Proton-c Null Messages
On Mon, Jun 08, 2015 at 01:47:03PM -0700, logty wrote: > The strange thing is pn_messenger_get is returning 0, I have code to throw an > error if it isn't 0. Also, the large messages work fine on RabbitMQ but not > on Apache Apollo. What's the output when you run your client with: $ PN_TRACE_FRM=1 [your client cmdline] ? Do you see the message being received on the client side? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpxSErwfJzge.pgp Description: PGP signature
Re: Proton-c Null Messages
On Mon, Jun 08, 2015 at 05:56:14AM -0700, logty wrote: > Hi all, > > I just had a quick question about proton-c returning null messages. I am > calling pn_messenger_get on my message and on my messenger and getting back > messages that are null. When I call: > > pn_bytes_t b = pn_data_get_bytes(pn_message_body(message)); > > calls to b.start and b.size create segfaults. Any ideas as to why this might > be happening? At the moment I am running a test with large messages (several > megabytes) being sent to the server. > > Thanks for the help! Are you *certain* you're getting messages back? What's the return value on the call to pn_messenger_get? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpq1f55F5Ztp.pgp Description: PGP signature
Re: Strange behaviour for pn_messenger_send on CentOS 6
On Thu, Jun 04, 2015 at 03:12:51PM +0100, Frank Quinn wrote: > I hit a strange issue today when setting up a qpid proton development > environment on a fresh CentOS 6 VM. I first found the issue in our > application, but when I went a little deeper, I realized I could recreate > the issue with the qpid proton send and recv example applications. All you > need to do is run ‘send’ on its own and the pn_messenger_send call hangs > indefinitely. If you start ‘recv’ first, it works fine, but ‘send’ on its > own hangs every time. > > This is contrary to its behaviour on my Fedora 21 laptop (latest yum > provisioned 0.8 version) where it always attempts once, logs a failure, > then exits (which is what I would expect). > > This effectively deadlocks our application. So far, I’ve tried compiling > qpid proton c myself (both 0.8 and 0.9.1), setting pn_messenger_send > timeout to 1 (it was previously -1), turning off iptables entirely and > disabling selinux and rebooting but no luck. Is this something you folks > have seen before? Hrm, this isn't something I've heard reported before. Does it do the same if you use the Python recv.py example as well? Also, can you do the following: $ PN_TRACE_FRM=1 ./recv [options] and share the output displayed? Also, is this solely with binaries you've built or are you installed RPMs from EPEL for Proton? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpMIyo8RW3eT.pgp Description: PGP signature
[jira] [Resolved] (PROTON-898) Ruby Messenger using pn_selectable_fd and not pn_selectable_get_fd
[ https://issues.apache.org/jira/browse/PROTON-898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-898. - Resolution: Fixed Fix Version/s: 0.10 > Ruby Messenger using pn_selectable_fd and not pn_selectable_get_fd > -- > > Key: PROTON-898 > URL: https://issues.apache.org/jira/browse/PROTON-898 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.10 > > > The API changed in the C code but the Ruby bindings have not been updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-897) Enhance the Ruby examples
[ https://issues.apache.org/jira/browse/PROTON-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-897. - Resolution: Fixed Fix Version/s: 0.10 > Enhance the Ruby examples > - > > Key: PROTON-897 > URL: https://issues.apache.org/jira/browse/PROTON-897 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.10 > > > Add a broker example application and standardize the names to reflect which > API set they represent; i.e., message_recv.rb, messenger_send.rb and > messenger_broker.rb. > Add a readme file that explains how to use each of the examples, showing > output and giving a brief explanation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-799) Provide the engine APIs in Ruby
[ https://issues.apache.org/jira/browse/PROTON-799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-799. - Resolution: Fixed Fix Version/s: 0.10 > Provide the engine APIs in Ruby > --- > > Key: PROTON-799 > URL: https://issues.apache.org/jira/browse/PROTON-799 > Project: Qpid Proton > Issue Type: New Feature > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.10 > > > Wrap the classes involved with the lower level driver APIs: > * driver > * listener > * connector > * transport > * connection > * sasl > * session > * event > * link > * delivery -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Pushing the Ruby Engine APIs to master...
It's been a while (14 days) since I pu up the pull request for the Ruby engine APIs [1]. If nobody has any objections, and since it won't affect anything outside of the ruby bindings, I plan to push this by COB tomorrow (Wednesday 03 June). [1] https://github.com/apache/qpid-proton/pull/33 -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpG10Go0KyYP.pgp Description: PGP signature
[jira] [Created] (PROTON-898) Ruby Messenger using pn_selectable_fd and not pn_selectable_get_fd
Darryl L. Pierce created PROTON-898: --- Summary: Ruby Messenger using pn_selectable_fd and not pn_selectable_get_fd Key: PROTON-898 URL: https://issues.apache.org/jira/browse/PROTON-898 Project: Qpid Proton Issue Type: Bug Components: ruby-binding Affects Versions: 0.9 Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce The API changed in the C code but the Ruby bindings have not been updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-897) Enhance the Ruby examples
Darryl L. Pierce created PROTON-897: --- Summary: Enhance the Ruby examples Key: PROTON-897 URL: https://issues.apache.org/jira/browse/PROTON-897 Project: Qpid Proton Issue Type: Improvement Components: ruby-binding Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce Add a broker example application and standardize the names to reflect which API set they represent; i.e., message_recv.rb, messenger_send.rb and messenger_broker.rb. Add a readme file that explains how to use each of the examples, showing output and giving a brief explanation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: New language bindings - when are they ready? [was Re: Ruby Engine APIs up for review]
On Mon, May 25, 2015 at 12:46:32PM -0400, Alan Conway wrote:
> On Thu, 2015-05-14 at 16:28 -0400, Darryl L. Pierce wrote:
> > I've pushed these APIs up for public review on their own branch [1] in
> > the Apache git repo. They're on a branch named ruby-engine-apis.
> >
> > Please take a look, tinker and please provide feedback. :D
> >
> > [1]
> > http://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;a=shortlog;h=refs/heads/ruby-engine-apis
>
> We have a bunch of new bindings on branches (Go, Ruby, C++) When is a
> binding ready for trunk?
>
> I would propose the following as a basic critiera:
>
> - Complete and tested for basic use.
> - API documentation in language-normal form (godoc, rdoc, doxgygen)
> - Core set of examples.
> - Automated testing of all examples.
> - Getting started README that explains how to run the examples.
>
> I would suggest the following core examples for all bindings:
>
> - send: connect to endpoint, send N messages.
> - receive: connect to endpoint, receive N messages.
> - broker: accept connections, in-memory queues created on demand.
> - request_response: synchronous request-response client.
> - echo_server: accept connections, echo message to reply-to address on
> same connection.
This sounds like a reasonable minimal set of examples to provide a POC
of the language.
Additionally, for each class of APIs (messaging, engine, reactive) we
should have matching examples as well: reactive_send, reactive_recv,
etc. This goes hand-in-hand with your later points WRT proper naming.
> This gives us a *small* set of examples that still has good coverage of
> key scenarios (brokered and client-server) and can be used *without* any
> external broker. This can be the common introduction to all our
> bindings.
>
> The python binding has all these, with different names. Rationale for
> proposing the above names:
>
> - The simplest send/receive examples should be called simply
> "send"/"receive". Python has "simple_send"/"simple_receive" which seems
> redundant.
> - "request_response" is more descriptive than
> "sync_client" ("sync_client" is a daft name, my bad)
> - "echo_server" describes what the server does, just "server" is to
> generic. The broker is also a server, and there may be other example
> servers.
>
> I can live with the python names if there's a consensus for that but now
> might be a good time to rationalize. I do think the python examples
> should be re-organized to start with these core examples. Right now the
> python README requires you to start by installing an unspecified
> "intermediary" which is going to cause confusion and frustration.
I was equally confused at first. When I see two examples with similar
names, like direct_send and direct_recv, my first thought was that the
two should be used together. So in addition to better names, the README
file(s) could groups together by topic to make it clearer how to use the
examples. Topics like "Simple Messaging", "Reactive Messaging", etc.
that notes the example app and how it works. I would find that easier to
follow and more intuitive than the current README.
> The very first README for every binding should show how to run
> send/receive examples against the example broker, and request_response
> against the example server. AFTER we have demonstrated that proton is
> useful *on its own* we can and should provide further examples of using
> it with other brokers, tornado, ruby/go/whatever specific frameworks and
> what have you.
>
> Some bindings (Go, possibly C++) provide additional API or toolkit to
> server multiple connections concurrently. They should also provide
> concurrent versions of the core examples. The go examples have send and
> receive take a list of addresses as arguments and send/receive on all
> addresses concurrently. I would suggest that as a model.
>
> Interested in feedback! I'll be working on Go and C++ in coming weeks so
> I'd like to co-ordinate with ruby and python efforts so we can present
> something coherent to our users.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
pgpsk7mgiqTLQ.pgp
Description: PGP signature
Re: Direct receive not responding in Ruby...
On Thu, May 21, 2015 at 04:37:09PM +0100, Gordon Sim wrote: > In python, it is the EndpointStateHandler, that is part of MessagingHandler > which is responsible for that. If ruby is doing something similar, I'd start > by checking if the equivalent handler is being called. As always you nailed it on the head. I'm in your debt, Gordon. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpyVksnSX8EE.pgp Description: PGP signature
Direct receive not responding in Ruby...
Got an interesting bug I'm digging into currently. In Ruby I've written analogs to the direct_* and simple_ send/recv example apps. The simple versions all work well and can interact with Python as expected. However, the direct_recv.rb example isn't. When the container starts listening for incoming direct connections, I see the following frames come in when the simple_send.rb client (or Python analog) connects: <- @open(16) <- @begin(17) <- @attach(18) But what I don't see is the outgoing connection connections that occurr in the Python version: -> @open(16) -> @open(17) -> @open(18) So the client connects but the flow never begins between the two. I'm digging into the acceptor code but don't see where Ruby is behaving differently from Python. Any suggestions? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp4bxwIzS6qx.pgp Description: PGP signature
[jira] [Resolved] (PROTON-883) Return the raw bytes from a transport buffer in Ruby.
[ https://issues.apache.org/jira/browse/PROTON-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-883. - Resolution: Fixed Fix Version/s: 0.9.1 > Return the raw bytes from a transport buffer in Ruby. > - > > Key: PROTON-883 > URL: https://issues.apache.org/jira/browse/PROTON-883 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.9.1 > > Attachments: 0001-PROTON-883-Wrap-pn_transport_peek-for-Ruby.patch > > > When retrieving a message that contains raw bytes rather than strings, the > Swig bindings can return a truncated string. This is due to the method > SWIG_FromCharPtr stopping at the first 0 byte since it's treating the content > as a C string and not as an array of bytes. > The code should, instead, using something like SWIG_FromCharPtrAndSize to > return the full content and not guess at the length based on null bytes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Ruby Engine APIs up for review
I've pushed these APIs up for public review on their own branch [1] in the Apache git repo. They're on a branch named ruby-engine-apis. Please take a look, tinker and please provide feedback. :D [1] http://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;a=shortlog;h=refs/heads/ruby-engine-apis -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpIVhSmU6rV8.pgp Description: PGP signature
[jira] [Updated] (PROTON-883) Return the raw bytes from a transport buffer in Ruby.
[ https://issues.apache.org/jira/browse/PROTON-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-883: Summary: Return the raw bytes from a transport buffer in Ruby. (was: Return the raw bytes as an array for a message in Ruby) > Return the raw bytes from a transport buffer in Ruby. > - > > Key: PROTON-883 > URL: https://issues.apache.org/jira/browse/PROTON-883 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Attachments: 0001-PROTON-883-Wrap-pn_transport_peek-for-Ruby.patch > > > When retrieving a message that contains raw bytes rather than strings, the > Swig bindings can return a truncated string. This is due to the method > SWIG_FromCharPtr stopping at the first 0 byte since it's treating the content > as a C string and not as an array of bytes. > The code should, instead, using something like SWIG_FromCharPtrAndSize to > return the full content and not guess at the length based on null bytes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-883) Return the raw bytes as an array for a message in Ruby
[ https://issues.apache.org/jira/browse/PROTON-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-883: Attachment: 0001-PROTON-883-Wrap-pn_transport_peek-for-Ruby.patch > Return the raw bytes as an array for a message in Ruby > -- > > Key: PROTON-883 > URL: https://issues.apache.org/jira/browse/PROTON-883 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Attachments: 0001-PROTON-883-Wrap-pn_transport_peek-for-Ruby.patch > > > When retrieving a message that contains raw bytes rather than strings, the > Swig bindings can return a truncated string. This is due to the method > SWIG_FromCharPtr stopping at the first 0 byte since it's treating the content > as a C string and not as an array of bytes. > The code should, instead, using something like SWIG_FromCharPtrAndSize to > return the full content and not guess at the length based on null bytes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Development workflow and release process [WAS: Concurrent Go API for proton is, erm, GO!]
On Tue, May 12, 2015 at 07:06:19PM -0400, Alan Conway wrote: > On Thu, 2015-05-07 at 15:42 -0400, Rafael Schloming wrote: > > > > 1. For developing new language bindings (and really for any development > > > > work that will involve enough new stuff to have a noisy commit history) > > > we > > > > use branches. This is already the case with the Ruby/C++/Python3 > > > bindings, > > > > as well as the SASL work. > > IMO this is a good policy. Go work is now on branch `go`. I left > bindings/go/README.md with a pointer to the new branch so people > following older discussions can find it. > > > I think I pretty much said go ahead on master, so apologies for singling > > you out. > "go" ahead. Hahaha. Just when I think we've done all those jokes, > there's always another one. I do like this. I would love to push my Reactive changes up to a ruby-reactive branch on the core repo and let people examine it there. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpfzdxQl8p7e.pgp Description: PGP signature
[jira] [Commented] (PROTON-738) Debian + Ubuntu Packages need updating to 0.8
[ https://issues.apache.org/jira/browse/PROTON-738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541902#comment-14541902 ] Darryl L. Pierce commented on PROTON-738: - The Proton 0.9 packages for Ubuntu 12 and 14 are available in the testing PPA: https://launchpad.net/~mcpierce/+archive/ubuntu/qpid-testing/+packages If you can verify them as working I'll be glad to promote them to the Qpid released PPA. I've submitted the 0.9 package to Debian and am waiting for my mentor to promote them. > Debian + Ubuntu Packages need updating to 0.8 > - > > Key: PROTON-738 > URL: https://issues.apache.org/jira/browse/PROTON-738 > Project: Qpid Proton > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Dominic Evans >Assignee: Darryl L. Pierce >Priority: Minor > > It would be great if we could get 0.8 packages pushed to the development > releases for debian and ubuntu > https://tracker.debian.org/pkg/qpid-proton > https://launchpad.net/ubuntu/+source/qpid-proton > And debs for the LTS releases pushed to the PPA > https://launchpad.net/~qpid/+archive/ubuntu/released -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Introducing the Ruby Reactive APIs
On Tue, May 12, 2015 at 04:10:54PM -0400, Darryl L. Pierce wrote: > On Tue, May 12, 2015 at 11:12:40AM -0400, Rafael Schloming wrote: > > I made a bunch of line comments. > > Things appear to be working in the scaled back example except for one > thing: when retrieving the record with pn_record_get the Swig wrapper > returned is of type SWIG:TYPE_p_void rather than the expected wrapper > for pn_rbkey_t. I think that Swig is confused by the type being returned > by pn_record_get and can't distinguish what the appropriate wrapper > should be. > > When the return value is then used for fetching the key, it fails with: > > TypeError: Expected argument 0 of type pn_rbkey_t *, but got > SWIG::TYPE_p_void #http://www.redhat.com/promo/vendor/ pgpNSkhYEFc0V.pgp Description: PGP signature
Re: Introducing the Ruby Reactive APIs
On Tue, May 12, 2015 at 11:12:40AM -0400, Rafael Schloming wrote: > I made a bunch of line comments. Things appear to be working in the scaled back example except for one thing: when retrieving the record with pn_record_get the Swig wrapper returned is of type SWIG:TYPE_p_void rather than the expected wrapper for pn_rbkey_t. I think that Swig is confused by the type being returned by pn_record_get and can't distinguish what the appropriate wrapper should be. When the return value is then used for fetching the key, it fails with: TypeError: Expected argument 0 of type pn_rbkey_t *, but got SWIG::TYPE_p_void #http://www.redhat.com/promo/vendor/ pgpCohtKHXqb9.pgp Description: PGP signature
Re: When pushing to the git repo, please avoid merge commits...
On Tue, May 12, 2015 at 02:10:59PM -0400, Alan Conway wrote: > On Mon, 2015-05-11 at 09:36 -0400, Darryl L. Pierce wrote: > master-branch $ git merge --ff-only task-branch > > That will refuse to merge unless the merge is a "fast-forward", which is > the simple "copy my commits to the end of trunk" that we want. It is > just a safety check - if you do what Darryl says it has no effect, but > if you forgot to rebase properly git will complain instead of creating > an unintended merge commit. +1 I've worked on a project that flat out rejects any commits that aren't FF. And it was very simple and clear to do a git --blame to look up where changes came from. > > http://mcpierce.blogspot.com/2012/08/git-fixup-and-autosquash.html > > > autosquash! Lovely! Git is full of nice surprises. When I learned the core set of tricks I use for working (interactive rebase, autofix/autosquash, bisect, cherry picking) is when I finally felt I was working with something more than just a version control system, but a power tool. :D -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpPw4RchyTEz.pgp Description: PGP signature
Re: Introducing the Ruby Reactive APIs
On Tue, May 12, 2015 at 11:12:40AM -0400, Rafael Schloming wrote: > I made a bunch of line comments. Excellent, thanks for that. I'll update the code and reply with my results. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp4ODW9uN9H0.pgp Description: PGP signature
Re: Introducing the Ruby Reactive APIs
On Tue, May 12, 2015 at 09:44:41AM -0400, Rafael Schloming wrote: > On Tue, May 12, 2015 at 8:34 AM, Darryl L. Pierce > wrote: > > > On Tue, May 12, 2015 at 05:45:20AM -0400, Rafael Schloming wrote: > > > Can you post an isolated reproducer with just your definition of > > pn_rbkey_t > > > and a code version of the 5 steps that lead to the seg fault? > > > > On my PROTON-781-reactive-ruby-apis branch is an example named > > "$REPO/examples/ruby/registry_test.rb" which does it. It's very pared > > down, only > > creating 3 Transport instances and consistently produces the segfault. > > > > That branch has about a 15 thousand line delta from master. That's a lot of > lines of code to hide a subtle memory bug. The pn_rbkey_t definition and > enough ruby code to use it in a proof of concept should only require a few > hundred line delta from master. I suggest producing just this delta for two > reasons. 1) Just the act of producing it will help narrow down where the > bug is, and 2) it gives me a much smaller delta to look at so I can be more > useful to you. > > (I did run the registry_test.rb that you pointed to through a debugger, > however its not obvious what the issue is upon inspection of the trace, and > while the ruby.i delta is pretty self contained, the details of how you are > using it from ruby are somewhere buried in the 15K diff.) I've pulled the pertinent pieces into a new branch: http://github.com/mcpierce/Proton/tree/rbkey-isolation The changes are all here: http://github.com/mcpierce/Proton/commit/2dc770992a7c02fe2054a4a77325a199e39b9c94 and it blows up exactly as I've been experiencing. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgppsp4k7wEmA.pgp Description: PGP signature
Re: Introducing the Ruby Reactive APIs
On Tue, May 12, 2015 at 05:45:20AM -0400, Rafael Schloming wrote: > Can you post an isolated reproducer with just your definition of pn_rbkey_t > and a code version of the 5 steps that lead to the seg fault? On my PROTON-781-reactive-ruby-apis branch is an example named "$REPO/examples/ruby/registry_test.rb" which does it. It's very pared down, only creating 3 Transport instances and consistently produces the segfault. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpPzg3g1inst.pgp Description: PGP signature
Re: Introducing the Ruby Reactive APIs
On Thu, May 07, 2015 at 01:02:13PM -0400, Rafael Schloming wrote: > The way the python code does this is by checking whenever a C object is > returned to python code. If the record contains an attachment indicating > that the C object has previously been wrapped, it uses this to > construct/retrieve an appropriate wrapper object. If it doesn't have the > appropriate attachment then it uses the record API to define/set the > attachment to the appropriate value. I presume you could do something > similar with ruby. After we chatted the other day, I've tried the following approach, using the pn_transport_t type as my test bed since it has relatively fewer dependencies. However, the plumbing in Proton for objects isn't quite clear to me and my code's not quite working the way we had discussed and I'm not sure why. The goal is to have a type that will live for as long as one of the impls in Proton lives; i.e., when we create something like pn_transport_t, the attachment created for this would hold some type that will get finalized when the pn_transport_t type is finalized. And that type would be the hook to clean up the single instance of a Ruby class that wraps the underlying C type. I've created a new type in the ruby.i descriptor for Swig and named it pn_rbkey_t, with three files: void* registry (a pointer to an object held in Ruby), char* method (the name of a method to invoke on that object), and char* key_value (the argument to be passed to that method). The code defines pn_rbkey_initialize and pn_rbkey_finalize methods, as well as getter and setter methods for the three fields. But I've put debugging into the code and never see the pn_rbkey_finalize method being invoked. My registry_test app does the following: 1. create an instance of pn_transport_t: impl = Cproton.pn_transport 2. create a Ruby Transport object: transport = Transport.wrap(impl) a. puts a weak reference to the Transport into the hashtable b. creates a pn_rbkey_t object and sets it as the sole record for the pn_transport_t object c. calls Cproton.pn_incref on the pn_rbkey_t instance 3. remove the reference: transport = nil 4. call garbage collection: ObjectSpace.garbage_collect 5. get the object back: transport = Transport.wrap(impl) a. calls pn_transport_attachment and retrieves the record created in 2 b. should then be able to get the key_value from the pn_rbkey_t type c. should then get the object out of the hashtable to return It's at step 5 that the example app segfaults. The segfault happens when, from Ruby, there's a call to print the attachment retrieve in 5a. Swig isn't failing since it's returning a value that seems to have been cached. But when Swig tries to retrieve data from the pn_rbkey_t struct underneath of it, *THAT* seems to have been reaped by Proton and Swig then segfaults, thinking there was an object still under the covers. Any ideas or suggestions of where to look for what's going on? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpmyZ1IC9zbA.pgp Description: PGP signature
Re: When pushing to the git repo, please avoid merge commits...
On Mon, May 11, 2015 at 09:23:27AM -0400, Darryl L. Pierce wrote: > A request for all, please avoid doing merge commits on your Git repo. > When these hit the shared repo it can cause Git to try and "fix" [1] the > commit when rebasing. > > The way I do commits with Git is to rebase my work branch on master > before merging it into master. That way it's a true merge and not a > merge commit. To do this: > > 1. Rebase your work branch: > > task-branch $ git rebase -i master > > 2. Fix any merge issues on the work branch until you get a clean set of >commits. > 3. Switch to master and then merge your branch: > > task-branch $ git checkout master > master-branch $ git merge task-branch > > Thanks. > > > [1] On my task branches I use autofix to merge changes back to the > appropriate previous commit in my branch. When I'm ready to merge them I > use "git rebase -i HEAD~[some # of commits] --autofix". Git then takes > all of the autofix commits and puts them after the appropriate commit to > merge together. > > If I a merge commit is accidentally included in there then git will try > to rebase that merge commit and all hell can break loose. Dug out this old post to my blog that has my steps for doing the commits on a task-based branch: http://mcpierce.blogspot.com/2012/08/git-fixup-and-autosquash.html I've added a new comment on this post with the steps for doing an autofix commit, but will write up a newer version of the post to include this inline. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp_NkrC53d2E.pgp Description: PGP signature
When pushing to the git repo, please avoid merge commits...
A request for all, please avoid doing merge commits on your Git repo. When these hit the shared repo it can cause Git to try and "fix" [1] the commit when rebasing. The way I do commits with Git is to rebase my work branch on master before merging it into master. That way it's a true merge and not a merge commit. To do this: 1. Rebase your work branch: task-branch $ git rebase -i master 2. Fix any merge issues on the work branch until you get a clean set of commits. 3. Switch to master and then merge your branch: task-branch $ git checkout master master-branch $ git merge task-branch Thanks. [1] On my task branches I use autofix to merge changes back to the appropriate previous commit in my branch. When I'm ready to merge them I use "git rebase -i HEAD~[some # of commits] --autofix". Git then takes all of the autofix commits and puts them after the appropriate commit to merge together. If I a merge commit is accidentally included in there then git will try to rebase that merge commit and all hell can break loose. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpfWWtjMVeGY.pgp Description: PGP signature
[jira] [Created] (PROTON-883) Return the raw bytes as an array for a message in Ruby
Darryl L. Pierce created PROTON-883: --- Summary: Return the raw bytes as an array for a message in Ruby Key: PROTON-883 URL: https://issues.apache.org/jira/browse/PROTON-883 Project: Qpid Proton Issue Type: Improvement Components: ruby-binding Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce When retrieving a message that contains raw bytes rather than strings, the Swig bindings can return a truncated string. This is due to the method SWIG_FromCharPtr stopping at the first 0 byte since it's treating the content as a C string and not as an array of bytes. The code should, instead, using something like SWIG_FromCharPtrAndSize to return the full content and not guess at the length based on null bytes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Introducing the Ruby Reactive APIs
On Thu, May 07, 2015 at 11:32:33AM -0400, Rafael Schloming wrote: > On Thu, May 7, 2015 at 10:40 AM, Darryl L. Pierce > wrote: > > > On Thu, May 07, 2015 at 09:57:49AM -0400, Rafael Schloming wrote: > > > On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce > > wrote: > > > > > > To help with this, two additional callback APIs were added to the > > Proton > > > > libraries: pn_record_set_callback and pn_record_has_callback. These two > > > > functions work to help register a method to be called whenever a record > > > > is deleted to enable memory management. This way the above-mentioned > > key > > > > can be properly deleted, and the value stored in the hash table > > > > discarded. > > > > > > I would need to see the code in detail, but I suspect you don't need to > > add > > > a pn_record_set_callback/get_callback to achieve roughly the > > functionality. > > > I *think* you could simply define a pn_class_t that is a reference > > counted > > > holder of your key. You could then put your callback logic in the > > finalizer > > > for that class, and when proton's reference counting triggers the > > > finalizer, it will run the callback logic at the appropriate time. > > > > (edit) > > > > As I was writing up a description of the code I realized I have already > > done what you suggest above WRT the pni_rbhandler_t type. I could use > > the same logic to create a pni_rbrecord_t type and manage its lifecycle > > the same way the handler's lifecycles are managed, yeah? > > Yes, I believe so. Since records are created when a struct if initially created, I'm not sure how to go about attaching the key to its lifecycle since the dynamic language isn't explicitly creating the record. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpPrDtwTjlAX.pgp Description: PGP signature
Re: Introducing the Ruby Reactive APIs
On Thu, May 07, 2015 at 09:57:49AM -0400, Rafael Schloming wrote: > On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce wrote: > > To help with this, two additional callback APIs were added to the Proton > > libraries: pn_record_set_callback and pn_record_has_callback. These two > > functions work to help register a method to be called whenever a record > > is deleted to enable memory management. This way the above-mentioned key > > can be properly deleted, and the value stored in the hash table > > discarded. > > I would need to see the code in detail, but I suspect you don't need to add > a pn_record_set_callback/get_callback to achieve roughly the functionality. > I *think* you could simply define a pn_class_t that is a reference counted > holder of your key. You could then put your callback logic in the finalizer > for that class, and when proton's reference counting triggers the > finalizer, it will run the callback logic at the appropriate time. (edit) As I was writing up a description of the code I realized I have already done what you suggest above WRT the pni_rbhandler_t type. I could use the same logic to create a pni_rbrecord_t type and manage its lifecycle the same way the handler's lifecycles are managed, yeah? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpFxlAV42Im8.pgp Description: PGP signature
Introducing the Ruby Reactive APIs
I've been working on this codebase since the beginning of the year. The two branches [1, 2] in my git repo represent the low-level engine APIs and the higher-level reactive APIs, respectively. I'm still working through the set of example apps for the reactive APIs, but at this point I feel this is close enough that I want to start getting feedback from people. == Memory Concerns Of particular important is memory management: the Proton libraries use reference counting to manage object lifespans, while Ruby uses mark and sweep operations for garbage collection. So ensuring that pure Ruby objects aren't reaped when they've only known to the Proton libraries, in the case of event handlers specifically, has been a challenge and one that's sure to have some cases that need fixing. The first model explored was to attachment the Ruby wrapper objects to the Swig-generated wrappers for the underlying C structs in Proton. Which worked at first, but turned out to be not useful. The reason being that the Swig bindings were themselves being reaped when they went out of scope; i.e., Swig doesn't maintain them by providing a mark operation until disposal of the underlying C structs. So this path, while initially promising, was discarded. The current model uses a hash table that is attached to the Qpid::Proton module. When objects are stored for use by the C libraries, they are tucked away in this hash table with a unique key generated based on memory addresses. A copy of that key, as a char*, is given to Proton to use later when the object is being retrieved. To help with this, two additional callback APIs were added to the Proton libraries: pn_record_set_callback and pn_record_has_callback. These two functions work to help register a method to be called whenever a record is deleted to enable memory management. This way the above-mentioned key can be properly deleted, and the value stored in the hash table discarded. The reference counting aspect of the Proton libraries is a concern as well. The code currently increments and decrements references in the same places as the Python code, but there are likely more places where such reference accounting need to be added. [1] http://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis [2] http://github.com/mcpierce/Proton/tree/PROTON-781-reactive-ruby-apis -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpXevt2WqzQw.pgp Description: PGP signature
[jira] [Resolved] (PROTON-873) Convert use of Object.send to Object.__send__ for Ruby bindings
[ https://issues.apache.org/jira/browse/PROTON-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-873. - Resolution: Fixed Fix Version/s: 0.10 > Convert use of Object.send to Object.__send__ for Ruby bindings > --- > > Key: PROTON-873 > URL: https://issues.apache.org/jira/browse/PROTON-873 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-873) Convert use of Object.send to Object.__send__ for Ruby bindings
Darryl L. Pierce created PROTON-873: --- Summary: Convert use of Object.send to Object.__send__ for Ruby bindings Key: PROTON-873 URL: https://issues.apache.org/jira/browse/PROTON-873 Project: Qpid Proton Issue Type: Improvement Components: ruby-binding Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-872) Ruby Messenger should not use send as it conflicts with Ruby's Object::send method
[ https://issues.apache.org/jira/browse/PROTON-872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-872. - Resolution: Not A Problem Ruby doesn't recommend using send, so using that name should not be a concern. > Ruby Messenger should not use send as it conflicts with Ruby's Object::send > method > -- > > Key: PROTON-872 > URL: https://issues.apache.org/jira/browse/PROTON-872 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > > Ruby's Object class has a method named "send" that is used for > programmatically invoking methods on an instance of a class. Classes should > not use "send" but instead use a different name for such methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-872) Ruby Messenger should not use send as it conflicts with Ruby's Object::send method
[ https://issues.apache.org/jira/browse/PROTON-872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14519918#comment-14519918 ] Darryl L. Pierce commented on PROTON-872: - Very good point. Thanks for the link, Rafi. I'll update the Reactive code to use __send__ instead of send. > Ruby Messenger should not use send as it conflicts with Ruby's Object::send > method > -- > > Key: PROTON-872 > URL: https://issues.apache.org/jira/browse/PROTON-872 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.9 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > > Ruby's Object class has a method named "send" that is used for > programmatically invoking methods on an instance of a class. Classes should > not use "send" but instead use a different name for such methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-872) Ruby Messenger should not use send as it conflicts with Ruby's Object::send method
Darryl L. Pierce created PROTON-872: --- Summary: Ruby Messenger should not use send as it conflicts with Ruby's Object::send method Key: PROTON-872 URL: https://issues.apache.org/jira/browse/PROTON-872 Project: Qpid Proton Issue Type: Bug Components: ruby-binding Affects Versions: 0.9 Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce Ruby's Object class has a method named "send" that is used for programmatically invoking methods on an instance of a class. Classes should not use "send" but instead use a different name for such methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Commented] (PROTON-344) Need Ruby version of msgr-send/msgr-recv tools
On Tue, Apr 28, 2015 at 07:51:06PM +, ASF subversion and git services (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/PROTON-344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517884#comment-14517884 > ] > > ASF subversion and git services commented on PROTON-344: > > > Commit 5f06462f37fa10d2da36e7755fa813c5b57986bd in qpid-proton's branch > refs/heads/master from [~astitcher] > [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=5f06462 ] > > PROTON-344: Fix code not to trigger pedantic C++ warning I think some commits are being applied to the wrong JIRA. I see a few commits against PROTON-344 (which is requestion a Ruby send/receive tool (which exists) but they're all from Andrew Stitcher for SASL work. I think they might be for PROTON-334 instead of -344. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp6pYERYOrBh.pgp Description: PGP signature
Strange memory issues with handlers
Seeing a strange issue while debugging a segfault issue with reactors
and Ruby. I'm seeing the char* value for a record showing up as the
dispatch address for either a handler->dispatch or else one of its
children.
= Records
In Ruby, to avoid objects being reaped by GC, they aren't given to the C
library. Instead, a char* is malloced and the string snprintf'd with the
hex address of the pn_record_t, the pn_handle_t and the value being
stored. That string is given to the pn_record_set to store, while a ruby
version of the string value is created and used as the key to stuff the
actual Ruby object into a hidden global hash.
Later, when pn_record_get is called, it retrieves that previous key
value and uses it to retrieve the object from the global hash.
Lastly, if registered, a cleanup function is called that allows for
deleting the malloc'd key, which also removes the entry from the hash
and allows the Ruby string and object to be GC'd.
= Handlers And The Memory Issue
So, like with the above records API, Ruby handlers can't be stored in
the C libraries directly. Instead, the code uses the struct:
typedef struct {
char *handler_key;
} pni_rbhandler_t;
The value for handler_key is, like with records, the hex form of the
Ruby object and the pni_rbhandler_t object created. And the handler is
stored in the global hash using that key value so it can be retrieved
later. And the pn_handler_finalize method removes the key and value from
the hash later.
= The Problem
In diagnosing the segfault I found that, when it happens, about 80% of
the time the segfault is caused by handler->dispatch having a bad value.
Specifically I saw values like this for the memory address held:
3536373137302d393231
Looking it, it looked like an ASCII string. So converting it to ASCII I
saw that it was, in this case, the string "567170-921". And when
something like that happened, there was a record stored previous that
had a key containing the string "129-071765"; i.e., the address pointed
to by handler->dispatch was overwritten by the string that had been
stored by a call to pn_record_set in a separate patch of code.
But the problem is that nothing outside of the library has direct access
to assign these values to fields, and I haven't quite tracked down where
the leak is in the code.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
pgpwv4XvpFkae.pgp
Description: PGP signature
Re: [VOTE]: Proton 0.9-rc-3
On Tue, Mar 17, 2015 at 09:42:18AM +1300, Rafael Schloming wrote: > Hi Everyone, > > Here's a quick respin of 0.9-rc-3. The only changes from rc-2 are exactly > those two mentioned on the rc-2 vote thread. I've included them at the end > for reference. You can find the source artifacts in the usual location: > > https://people.apache.org/~rhs/qpid-proton-0.9-rc-3/ > > Java binaries are here: > > https://repository.apache.org/content/repositories/orgapacheqpid-1031 > > Please check it out and register your vote: > > [ X ] Yes, release Proton 0.9-rc-3 as 0.9 final > [ ] No, because ... Packaged for Fedora. Installed all packages. Tested across language bindings (Python, Perl, Ruby). Found a single non-blocking issue in Perl that I've fixed and pushed. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpWRFD4rhjeD.pgp Description: PGP signature
Re: VOTE: Release Proton 0.9-rc-1 as 0.9 final
On Mon, Mar 09, 2015 at 09:13:07AM -0400, Ken Giusti wrote: > Anyone else getting the following errors when building the docs? > > > ;; This buffer is for notes you don't want to save, and for Lisp evaluation. > ;; If you want to create a file, visit that file with C-x C-f, > ;; then enter the text in that file's own buffer. > > Generating example index... > finalizing index lists... > lookup cache used 974/65536 hits=10705 misses=982 > finished... > Built target docs-c > [ > Error: Could not find a file or object named proton/handlers.py > Error: Could not find a file or object named proton/reactor.py > Error: Could not find a file or object named proton/utils.py > Error: Could not find a file or object named proton/wrapper.py > [.. > + > | File > /home/kgiusti/work/proton/0.9rc1/qpid-proton-0.9-rc-1/proton-c/bindings/python/proton/__init__.py, > line 3778, in proton.Url > | Warning: Line 3781: Improper paragraph indentation. > | > > [] > Built target docs-py I'm seeing it on my end, yes. The build finishes, but the installed docs don't include the messaging package or anything outside of the proton module. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpMx07M9x1Wb.pgp Description: PGP signature
Re: Proton and Ruby object references...
On Fri, Mar 06, 2015 at 04:41:34PM -0500, Darryl L. Pierce wrote: > Revisiting this topic given newly discovered issues: Thinking about this over the weekend, it feels like it would be a huge nightmare to try and put a shim in place to keep a Ruby object held by a C struct from being reaped. Especially given that Swig is recreating Ruby wrappers and losing state information on them. But I had an idea taken from a previous project that feels like it would solve this issue. Instead of having a Proton C struct hold a reference to a Ruby object and dealing with the GC issues, why not instead have that C struct hold a unique string? Within Ruby we can maintain a private (not visible to the application) hashtable that uses that string value as a key to point to a Ruby object that is the true value of interest. To assist this, we would add a function pointer to the record APIs (pn_record_remove) that gets called when a record is deleted. It passes in the void* value for each field in the record, if the function is assigned, and that function can work as a hook to remove the object from the global hash. Why do this instead of having C hold a Ruby object and stuffing the Ruby object into some sort of array? I think it would be a simpler, cleaner operation to have C not know if's holding onto Ruby. So, for example, if something happened to the hash or the object held in the hash, we wouldn't get a segfault to access the value from C: instead, we would have a key with no matching value. Something that would be easier to debug than what could appear to be a random segfault. Thoughts? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpG_tP74PzQd.pgp Description: PGP signature
Proton and Ruby object references...
Revisiting this topic given newly discovered issues: Back in January I had a solution that worked quite well with passing in Ruby objects to the C library. Ruby objects could be attached to Swig wrappers around structs so that their lifecycle's were kept; i.e., when C object X was garbaged collected, its wrapper would go away, and associated objects Y & Z were reaped since they no longer had a parent. However, it's now come up that the Swig wrappers themselves don't seem to live a terribly long time in some cases. Also, asking Swig for the same struct more than once results in a completely different Swig wrapper coming back; i.e., there's no one-to-one mapping between the Swig wrapper and the underlying struct. And, the big problem, the Ruby object passed to the C structs are still vulnerable to being reaped as a result of this. To prove this, I created two test APIs to store and retrieve a single pn_transport_t instance [1]. I create a transport, store it and then fetch it again and get two different objects that are are not equal: t = Cproton.pn_transport => t now holds a reference to an instance of pn_transport_t Cproton.pn_transport_set_mine t => that pn_transport_t struct is now pointed to in the C libraries u = Cproton.pn_transport_get_mine => u now holds a reference to the same instance of pn_transport_t t == u => false -- comparison fails Additionally, the pn_record_get API seems to have an issue with retrieving stored objects that may or may not be related to this issue. I've written a reproducer [2] that simply stores and retrieves attachments for a transport, creates a hash, forces garbage collection and then shows object statistics for the MyHash and wrapper for the pn_record_t type in Ruby space. Specifically, if lines 101 and 103-4 are commented out, the reproducer runs fine and creates a whole lot of new transports before going back for the original and getting its attachments. If lines 102 thru 104 are commented out the reproducer runs fine and retrieves the same attachments over and over without a problem. BUT, if 101-2 are commented out and the reproducer alternates between getting the stored one and a new one, it blows up on the first call to retrieve the stored reference after getting a single new transport. This doesn't seem to make sense since I can create 1k new transports and then get my old one back, but creating a single new one and then getting the old one back it blows up. And where it blows up is on displaying the retrieved attrs object. I know that this issue is related to garbage collection: I can add the attrs variable to a global array and the error never happens in the last scenario. But this doesn't seem to be a good solution since it would require a lot of extra code to track and remove objects when a C struct is deleted. So, in the end, I need a better way to store and retrieve Ruby objects in the library than is currently being done. It works well with the engine APIs since you're not creating and then re-creating objects, so there's not a lot of storing and fetching going on. But with the reactive APIs, it wants to store object state details and recreate them via the various class wrap methods. And this is failing for Ruby since Ruby's mark and sweep is deleting objects that C is holding. [1] pn_transport_set_mine, pn_transport_get_mine [2] http://github.com/mcpierce/Proton/blob/PROTON-799-Ruby-engine-apis/examples/ruby/wrapper_test.rb -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpCQP9zPoED5.pgp Description: PGP signature
[jira] [Assigned] (PROTON-738) Debian + Ubuntu Packages need updating to 0.8
[ https://issues.apache.org/jira/browse/PROTON-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce reassigned PROTON-738: --- Assignee: Darryl L. Pierce > Debian + Ubuntu Packages need updating to 0.8 > - > > Key: PROTON-738 > URL: https://issues.apache.org/jira/browse/PROTON-738 > Project: Qpid Proton > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Dominic Evans > Assignee: Darryl L. Pierce >Priority: Minor > > It would be great if we could get 0.8 packages pushed to the development > releases for debian and ubuntu > https://tracker.debian.org/pkg/qpid-proton > https://launchpad.net/ubuntu/+source/qpid-proton > And debs for the LTS releases pushed to the PPA > https://launchpad.net/~qpid/+archive/ubuntu/released -- This message was sent by Atlassian JIRA (v6.3.4#6332)
pn_incref/pn_decref and Ruby
Do I need to use reference counting from within the Proton library even though I'm writing in a language that doesn't use them? I asked because I see calls to pn_incref/pn_decref in the Python bindings. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp5O0WzpfHeA.pgp Description: PGP signature
Ruby: Engine send experiencing problems
So I'm hitting an interesting problem now with the work I'm doing on the Ruby engine APIs. I have both engine_send.rb and engine_recv.rb [1] examples written and working well. Today I changed them to be single-threaded applications, and that's when things got interesting. With a single threaded sender and receiver, the receiver hand handle messages up to a point (from about 2400 to 4800 messages) before blowing up with a EPIPE error. In debugging this I added a small piece of output to show the state of the sender. And suddenly the problem disappeared! I comment out the this piece of debugging and the problem comes back. Talking with a cube neighbor, I asked for a suggestion from him. His idea was to add a small sleep (0.0001s) at the spot where the debug line was executed in the code. Problem goes away. Change that to "sleep 0". Problem goes away. Comment the line out, and the problem comes back. So it's a very interesting issue. Adding the sleep 0 still involves a kernel call, so there's still even the smallest of delays there. But it seems that, for some reason, the code seems to be overrunning the outgoing buffer (first theory) and causing the connection to close. If you look at the selectable.rb class [2] it reads as only pushing as many bytes into the transport as it says it can handle. *LAST MINUTE EDIT* When turning on debugging in the send app (--debug, which just outputs a single line displaying an event when it occurs) is enough to keep the problem from occurring. It feels like this problem might be a buffer having a problem, though whether it's on the send or the receive side I'm not sure yet. The Ruby code is based on the Python code for handling reading and writing, and it asks the underlying Transport how many bytes can be written to it before trying to do so. [1] https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis/examples/ruby [2] https://github.com/mcpierce/Proton/blob/PROTON-799-Ruby-engine-apis/examples/ruby/lib/selectable.rb -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgprHNmRNGs10.pgp Description: PGP signature
Re: Ruby, Proton Engine and Records
On Fri, Feb 06, 2015 at 08:37:07AM -0500, Rafael Schloming wrote: > Is this the approach you referenced in your other email? (Same request here > for a pointer to the highlighted key bits of the approach.) No, this was a third approach to the problem, since we can't directly affect the object created by Swig via Data_Wrap_Struct. You can see the specific changes to ruby.i here [1]. http://github.com/mcpierce/Proton/commit/1e66f39535e71dc376d69abde7b5741929d33a3d -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgprYbkagDgS7.pgp Description: PGP signature
[jira] [Resolved] (PROTON-775) ruby: message annotations send from a ruby client are invalid
[ https://issues.apache.org/jira/browse/PROTON-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-775. - Resolution: Fixed Fix Version/s: 0.9 > ruby: message annotations send from a ruby client are invalid > - > > Key: PROTON-775 > URL: https://issues.apache.org/jira/browse/PROTON-775 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.8 >Reporter: Dominic Evans > Assignee: Darryl L. Pierce > Fix For: 0.9 > > Attachments: > 0001-PROTON-775-Ensure-that-annotation-keys-in-Ruby-are-e.patch > > > Since PROTON-616, proton-j has attempted to enforce the fact that the key's > of a message annotations map *must* be of the Symbol type (or actually ulong, > but those are reserved so we don't need to worry about that) [1] > Unfortunately, from the Ruby client, the mapping of a native map into proton > data always sets the key's as strings rather than symbols. > In proton-j, you'll currently hit a `java.lang.ClassCastException: > java.lang.String incompatible with org.apache.qpid.proton.amqp.Symbol` if you > try to look at keys of the MessageAnnotations object of a message sent from > the ruby send.rb example (which sets message annotations for the keys > 'version' and 'pill'). > -- > [1] > http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-annotations > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-775) ruby: message annotations send from a ruby client are invalid
[ https://issues.apache.org/jira/browse/PROTON-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-775: Attachment: 0001-PROTON-775-Ensure-that-annotation-keys-in-Ruby-are-e.patch This patch changes how annotations are encoded to ensure that the keys are sent as symbols. > ruby: message annotations send from a ruby client are invalid > - > > Key: PROTON-775 > URL: https://issues.apache.org/jira/browse/PROTON-775 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.8 >Reporter: Dominic Evans > Assignee: Darryl L. Pierce > Attachments: > 0001-PROTON-775-Ensure-that-annotation-keys-in-Ruby-are-e.patch > > > Since PROTON-616, proton-j has attempted to enforce the fact that the key's > of a message annotations map *must* be of the Symbol type (or actually ulong, > but those are reserved so we don't need to worry about that) [1] > Unfortunately, from the Ruby client, the mapping of a native map into proton > data always sets the key's as strings rather than symbols. > In proton-j, you'll currently hit a `java.lang.ClassCastException: > java.lang.String incompatible with org.apache.qpid.proton.amqp.Symbol` if you > try to look at keys of the MessageAnnotations object of a message sent from > the ruby send.rb example (which sets message annotations for the keys > 'version' and 'pill'). > -- > [1] > http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-annotations > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-775) ruby: message annotations send from a ruby client are invalid
[ https://issues.apache.org/jira/browse/PROTON-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce reassigned PROTON-775: --- Assignee: Darryl L. Pierce > ruby: message annotations send from a ruby client are invalid > - > > Key: PROTON-775 > URL: https://issues.apache.org/jira/browse/PROTON-775 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.8 >Reporter: Dominic Evans > Assignee: Darryl L. Pierce > > Since PROTON-616, proton-j has attempted to enforce the fact that the key's > of a message annotations map *must* be of the Symbol type (or actually ulong, > but those are reserved so we don't need to worry about that) [1] > Unfortunately, from the Ruby client, the mapping of a native map into proton > data always sets the key's as strings rather than symbols. > In proton-j, you'll currently hit a `java.lang.ClassCastException: > java.lang.String incompatible with org.apache.qpid.proton.amqp.Symbol` if you > try to look at keys of the MessageAnnotations object of a message sent from > the ruby send.rb example (which sets message annotations for the keys > 'version' and 'pill'). > -- > [1] > http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-annotations > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-775) ruby: message annotations send from a ruby client are invalid
[ https://issues.apache.org/jira/browse/PROTON-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309115#comment-14309115 ] Darryl L. Pierce commented on PROTON-775: - [~dnwe] is this something we can fix in mapping.rb in MAP.put? If the keys for a map must be a Symbol we should be able to enforce it there, yeah? > ruby: message annotations send from a ruby client are invalid > - > > Key: PROTON-775 > URL: https://issues.apache.org/jira/browse/PROTON-775 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.8 >Reporter: Dominic Evans > > Since PROTON-616, proton-j has attempted to enforce the fact that the key's > of a message annotations map *must* be of the Symbol type (or actually ulong, > but those are reserved so we don't need to worry about that) [1] > Unfortunately, from the Ruby client, the mapping of a native map into proton > data always sets the key's as strings rather than symbols. > In proton-j, you'll currently hit a `java.lang.ClassCastException: > java.lang.String incompatible with org.apache.qpid.proton.amqp.Symbol` if you > try to look at keys of the MessageAnnotations object of a message sent from > the ruby send.rb example (which sets message annotations for the keys > 'version' and 'pill'). > -- > [1] > http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-annotations > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Non-reactive engine examples?
On Mon, Feb 02, 2015 at 09:12:22AM -0500, Darryl L. Pierce wrote: > Are there any example apps for Python that don't use the reactive APIs? If no examples, perhaps a primer of how the logical components of the enger interact to help guide providing an example? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpCxB5cYjxAI.pgp Description: PGP signature
Non-reactive engine examples?
Are there any example apps for Python that don't use the reactive APIs? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpBlkS2f_1Sx.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Wed, Jan 28, 2015 at 01:22:45PM -0500, Rafael Schloming wrote: > Also, have you been able to validate your testing strategy for either/both > of these POCs? Can you generate seg faults and/or valgrind warnings when > you intentionally comment out the line of code that keeps the reference > alive? The POC that uses manual wrapping of a C struct works correctly, preventing objects from being reaped without leaking memory. I validated this by creating exhaustive (1M+) instances of both pure Ruby and C structs that have been wrapped via the Data_Wrap_Struct, assigning the Ruby object to the C struct so that only C held a reference to it, then calling GC.start to reap objects and then checking that the expected number of the pure Ruby objects still existed, via ObjectSpace.each_object([class]).count. Accessing the C-help Ruby object and doing functions such as class_eval on it worked without segmentation faults. I then ensures that it wasn't a fluke by commenting out, in the function that marks the Ruby object in C to keep it from being reaped, and re-running the tests. The app *immediately* segfaults after trying to class_eval the first Ruby object after garbage collection. So this path is the right one to follow. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpjpMUrU8T1Y.pgp Description: PGP signature
Ruby, Proton Engine and Records
So over the last week I've been working on a way to avoid leaking memory or causing segfaults when the underlying C library is giving a reference to a Ruby object. And after much coding and gnashing of teeth I foudn the easiest way to do this with manually wrapping the pn_record_t struct. That would allow for adding a mark function to mark any Ruby objects so that they could not be reaped during garbage collection. And this would work fine but for one problem: the instances of pn_record are created by transport, etc. as part of *their* initializations. So there is no way for us to manually wrap the pn_record_t type for Ruby use in Swig. So looking at things, I have an alternate method that's not brittle and which gives us the feature we're looking for. From the Swig code, we can wrap the call to pn_record_set, at which point we have access to the Ruby wrapper for the pn_record_t type. We can then add/modify instance variables on that Ruby wrapper. Specifically, we can add and then update one named pn_record_attrs (or any other arbitrary name) and use it to track any objects that're added or replaced for the record. And the bonus is that, when the pn_record_t is deleted, it's Ruby wrapper is deleted as well and _all_ instance variables (including our Ruby objects) will be reaped along with it. I'm going to do some more testing on this, but my initial findings support this memory management scheme as viable. I have a test application that creates 500k+ values, storing them in a single record instance. Iterating back over the records and recovering them after running garbage collect returns variables as expected. Commenting out the portion that manages the wrapper instance variable and the same tests fail with a segfault while accessing the records. The work can all be checked out here [1], with the changes to the ruby.i file as the most recent commit in the branch. I'd love to get some feedback on this work in general as well. [1] https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp3CnIpetAP6.pgp Description: PGP signature
Questions regarding pn_record_* APIs
Can someone provide a little detail on the set of APIs? I'm looking to get a better understanding of the PYCTX value defined in the wrapper.py class and how it is uses in the APIs when passed to pn_record_def. I'm looking to emulate this behavior in the Ruby but am unclear as to what the PYCTX value represents. Any insight would be helpful. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpAyZ6OoOVeX.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Wed, Jan 28, 2015 at 01:10:28PM -0500, Rafael Schloming wrote: > On Wed, Jan 28, 2015 at 1:05 PM, Darryl L. Pierce > wrote: > > > On Wed, Jan 28, 2015 at 12:06:44PM -0500, Rafael Schloming wrote: > > > Why did you reject it then? > > > > Reject it? I don't recall rejecting any option. > > I meant why did you post about the global array thing and not this. Is > there some reason you think one is preferred to the other? I described it since it was the one different from what we discussed early based on the blog posts. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpldLHb8zHLD.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Wed, Jan 28, 2015 at 12:06:44PM -0500, Rafael Schloming wrote: > Why did you reject it then? Are you referring to this? "Though, I was hoping we could avoid having to manually do things..." What I meant was that I would like to keep the work within the confines of the Swig code. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp41pGyVkchT.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Wed, Jan 28, 2015 at 12:06:44PM -0500, Rafael Schloming wrote: > Why did you reject it then? Reject it? I don't recall rejecting any option. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp_t6ZoIcDDx.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Wed, Jan 28, 2015 at 09:19:29AM -0500, Rafael Schloming wrote: > On Wed, Jan 28, 2015 at 9:06 AM, Darryl L. Pierce > wrote: > > > On Wed, Jan 28, 2015 at 06:04:57AM -0500, Rafael Schloming wrote: > > > On the face of it this sounds like it could be quite brittle and probably > > > more complicated than just forgetting about swig for the one pn_rubyref_t > > > struct and wrapping it manually. Did you attempt the latter option at > > all? > > > > Yes, there is a POC of this on my branch as well. > > > > Did it work? Can you send me a pointer to it? Yes, it works. It uses Data_Wrap_Struct to encapsulate the pn_rubyref_t type, rb_gc_mark to mark any Ruby object held by that type against reaping, and does appropriate alloc and free operations on instances of the struct. https://github.com/mcpierce/Proton/tree/c-to-ruby-reference-gc-check -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpRloAgXb9mC.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Wed, Jan 28, 2015 at 06:04:57AM -0500, Rafael Schloming wrote: > On the face of it this sounds like it could be quite brittle and probably > more complicated than just forgetting about swig for the one pn_rubyref_t > struct and wrapping it manually. Did you attempt the latter option at all? Yes, there is a POC of this on my branch as well. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgphTZBbOghP3.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Fri, Jan 23, 2015 at 03:46:34PM -0500, Darryl L. Pierce wrote: > +1 Though, I was hoping we could avoid having to manually do things... So I have a working POC that assigns a Ruby object to a C struct in such a way as to keep the Ruby object from being reaped. The solution (for now) stores the object in a hidden global array for such objects for as long as they're held by the C structure and, when C is deleted or the reference changed, the object is removed from the array and available for reaping. I submitted a question to the Swig users mailing list, but that seems to be pretty low traffic and effectively unmanned ATM. Only 15 posts there in the last month and none of them have followups. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpAejYR2VHPv.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Fri, Jan 23, 2015 at 03:25:33PM -0500, Rafael Schloming wrote: > On Fri, Jan 23, 2015 at 2:08 PM, Darryl L. Pierce > wrote: > > > On Fri, Jan 23, 2015 at 01:49:34PM -0500, Rafael Schloming wrote: > > > It does talk about what swig does, but it also talks about the other > > > direction: > > > > > > "If the C structure references other ruby objects, then the mark function > > > pointer must also be provided and must properly mark the other objects > > with > > > rb_gc_mark()" > > > > What I meant is it's not showing how to do that. Swig is the one that's > > generating that Ruby struct wrapping, but I haven't found (yet) if they > > expose to users a way to tap into that adn do what we want. > > > > If you can find a way for swig to let you control it that's great, but I > was thinking we could just wrap this one struct by hand. It shouldn't be a > whole lot of code for just that struct, and swig can still handle the rest > of them. +1 Though, I was hoping we could avoid having to manually do things... -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpX0na7kqfSD.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Fri, Jan 23, 2015 at 01:49:34PM -0500, Rafael Schloming wrote: > It does talk about what swig does, but it also talks about the other > direction: > > "If the C structure references other ruby objects, then the mark function > pointer must also be provided and must properly mark the other objects with > rb_gc_mark()" What I meant is it's not showing how to do that. Swig is the one that's generating that Ruby struct wrapping, but I haven't found (yet) if they expose to users a way to tap into that adn do what we want. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpnb2qwpo8uP.pgp Description: PGP signature
Re: Ruby memory management (was: Ruby and the Engine APIs)
On Fri, Jan 23, 2015 at 11:02:59AM -0500, Rafael Schloming wrote: > For more info on how to integrate with Ruby's GC you can read this > article[1]. It's one of the few pieces of documentation I've found that > actually explain how to keep a reference from C to a Ruby object. > > [1] > http://clalance.blogspot.com/2013/11/writing-ruby-extensions-in-c-part-13.html This blog post shows how to manually do what Swig is doing for us: represent a C struct as something Ruby can touch, with hooks to release memory when GC runs on the Ruby wrapper. But, sadly, it's not showing what we need, which is how to assign a reference to a Ruby object to a C struct and then have Ruby not GC that Ruby object. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgptytdsND3ZS.pgp Description: PGP signature
Ruby memory management (was: Ruby and the Engine APIs)
On Thu, Jan 22, 2015 at 12:08:52PM -0500, Rafael Schloming wrote: > The most important thing to get worked out for this is the memory > management semantics between C and Ruby. From what I can tell from your > branch, it looks like you haven't done that yet. My initial readings into how Ruby handles object references initially lead me to believe that, with Swig, we're receiving some benefit already WRT object management. Since Ruby uses mark and sweep rather than reference counts, when a Ruby object (or an C object wrapped by Swig as a Ruby object) goes out of reference then it'll get garbage collected. But I won't lie: there might be some subtle detail I'm missing here. I played around with this last night after your suggestion and wrote some stuff on a side branch [1]. It's a simple test that uses the following types: * pn_rubyref_t - C type in Proton that's wrapped by Swig; holds a void * reference to any kind of object * Farkle - a pure Ruby class I then wrote a simple app that creates 1M instances of rb_rubyref_t and 1M instances of Farkle as assigns a Farkle to each rb_rubyref_t. It then puts them each into an array, which is size limited to 100k entries (to allow any GC to run while the app is going). I also added finalizer function to Farkle that just outputs some text when the object is being garbage collected. What I see as the app runs is the output of the Farkle instances being garbage collected while there are still instances being created. The app then ends by sleeping and then exiting. Before and after sleeping the app outputs the number of objects (via ObjectSpace) exist for each type. Running it multiple times I see all of the pn_rubyref_t and Farkle instances being cleaned up, no memory leaks. I then changed things to make a circular reference between Farkle and pn_rubyref_t. Re-ran the tests and still see the objects getting cleaned up. I also ran top to keep an eye on memory usage for the ruby-mri process. O_o (1) [J:0/1028] mcpierce@mcpierce-laptop:cmake (0.3-fedora) $ top -b | grep ruby-mri 19989 mcpierce 20 0 188964 23228 6380 S 53.3 0.3 0:00.55 ruby-mri 19989 mcpierce 20 0 202644 36892 6380 S 18.3 0.5 0:01.10 ruby-mri 19989 mcpierce 20 0 207492 41784 6380 R 19.9 0.5 0:01.70 ruby-mri 19989 mcpierce 20 0 227748 62108 6380 S 17.6 0.8 0:02.23 ruby-mri 19989 mcpierce 20 0 229796 64020 6380 R 22.3 0.8 0:02.90 ruby-mri 19989 mcpierce 20 0 233096 67188 6380 R 57.1 0.8 0:03.42 ruby-mri 19989 mcpierce 20 0 233096 67188 6380 S 0.7 0.8 0:03.44 ruby-mri 19989 mcpierce 20 0 233096 67188 6380 S 0.0 0.8 0:03.44 ruby-mri 19989 mcpierce 20 0 233096 67188 6380 S 0.0 0.8 0:03.44 ruby-mri 19989 mcpierce 20 0 233096 67188 6380 S 1.3 0.8 0:03.48 ruby-mri 19989 mcpierce 20 0 235340 69564 6380 R 48.5 0.9 0:04.94 ruby-mri 19989 mcpierce 20 0 235340 69564 6380 S 47.5 0.9 0:06.37 ruby-mri 19989 mcpierce 20 0 235340 69564 6380 R 50.5 0.9 0:07.89 ruby-mri 19989 mcpierce 20 0 235340 69564 6380 S 48.0 0.9 0:09.34 ruby-mri I only show those lines since, after that point, the virtual memory footprint didn't grow for the run os the apps run. [1] https://github.com/mcpierce/Proton/tree/c-to-ruby-reference-gc-check -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp0PMt9ATWz_.pgp Description: PGP signature
Re: removing the proton driver API
On Thu, Jan 22, 2015 at 10:50:22AM -0500, Rafael Schloming wrote: > I just noticed that dispatch seems to have it's own copy of driver.c now. I > think that means the driver API is now dead code as messenger, the new > reactor stuff, etc all use the newer selector API. > > Is anyone else using/aware of anyone using this code in anyway? I would > like to at a minimum deprecate it for 0.9, and preferably remove it > entirely if it is in fact currently unused. Are there other APIs, like listener or connector, that'll be removed as well? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgps7d72Xxvta.pgp Description: PGP signature
Ruby and the Engine APIs
I've been working on providing the low-level engine APIs in Ruby over the past few weeks, and from what I can see I think I'm near to completion regarding wrapping them. However, I'm not sure if I'm missing something since I don't see a way to use the code to create a connection. :D Specifically, what I want to do next is create examples of working with these low-level APIs, initially a simple send/receive example would be best with a receiver that listens for a connection and a sender that connects to it, all while being able to toggle tracing the communication, etc. I see nothing existing for other languages to use as a guide, so am a little stumped at the moment. The work I've done to date is here: https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpZ0NMIWLvW8.pgp Description: PGP signature
Re: c reactor / gordon's examples
On Wed, Jan 14, 2015 at 08:28:50AM -0500, Rafael Schloming wrote:
> import sys
>
> reactor = Reactor(Program())
> reactor.acceptor("0.0.0.0", "5672")
> reactor.connection(Client("localhost:5672", sys.argv[1:]))
> reactor.run()
I like the simplicity of how a simple app can be setup. The above can be
used inside of an EventMachine block in Ruby. Would we be able to pass a
file descriptor to the underlying C code? I'm wondering if we could
provide an AMQPServer similar to others provided [1].
[1] https://github.com/eventmachine/eventmachine/tree/master/lib/em/protocols
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
pgpJtcVpn667n.pgp
Description: PGP signature
[jira] [Created] (PROTON-799) Provide the engine APIs in Ruby
Darryl L. Pierce created PROTON-799: --- Summary: Provide the engine APIs in Ruby Key: PROTON-799 URL: https://issues.apache.org/jira/browse/PROTON-799 Project: Qpid Proton Issue Type: New Feature Components: ruby-binding Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce Wrap the classes involved with the lower level driver APIs: * driver * listener * connector * transport * connection * sasl * session * event * link * delivery -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-781) Implement the Reactive APIs in Ruby
[ https://issues.apache.org/jira/browse/PROTON-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255720#comment-14255720 ] Darryl L. Pierce commented on PROTON-781: - The Reactive APIs are an add-on to the Engine APIs in Proton. They provide a way of communicating event changes to an application, allowing it to react to such events as a connection being established, a remote link closing, or a message being received. Gordon has broken ground by providing an implementation of this in the Python libraries, and this JIRA regards taking that effort and bringing it to the Ruby APIs as well. If you look on master in the Proton library, you can see the work he's done in $REPO/proton-c/bindings/python/proton/ with guiding examples in $REPO/examples/engine/py/ > Implement the Reactive APIs in Ruby > --- > > Key: PROTON-781 > URL: https://issues.apache.org/jira/browse/PROTON-781 > Project: Qpid Proton > Issue Type: Improvement > Components: ruby-binding > Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-471) Expose the Messenger work method in Perl bindings
[ https://issues.apache.org/jira/browse/PROTON-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-471. - Resolution: Fixed Fix Version/s: 0.9 > Expose the Messenger work method in Perl bindings > - > > Key: PROTON-471 > URL: https://issues.apache.org/jira/browse/PROTON-471 > Project: Qpid Proton > Issue Type: New Feature > Components: proton-c > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.9 > > > This feature is not currently present. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-781) Implement the Reactive APIs in Ruby
Darryl L. Pierce created PROTON-781: --- Summary: Implement the Reactive APIs in Ruby Key: PROTON-781 URL: https://issues.apache.org/jira/browse/PROTON-781 Project: Qpid Proton Issue Type: Improvement Components: ruby-binding Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-779) Building Proton documentation fails due to missing proton.py file
[ https://issues.apache.org/jira/browse/PROTON-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-779. - Resolution: Fixed Fix Version/s: 0.9 > Building Proton documentation fails due to missing proton.py file > - > > Key: PROTON-779 > URL: https://issues.apache.org/jira/browse/PROTON-779 > Project: Qpid Proton > Issue Type: Bug > Components: python-binding >Affects Versions: 0.9 > Environment: RHEL 7 >Reporter: Irina Boverman > Assignee: Darryl L. Pierce > Fix For: 0.9 > > > cmake -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo > -DCMAKE_SHARED_LINKER_FLAGS="-Wl,-z,relro" -DCMAKE_INSTALL_PREFIX=/usr > -DSYSINSTALL_BINDINGS=ON -DBUILD_RUBY=OFF .. > make all docs > Results in this: > Built target docs-c > make -f proton-c/bindings/python/CMakeFiles/docs-py.dir/build.make > proton-c/bindings/python/CMakeFiles/docs-py.dir/depend > make[3]: Entering directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > cd /var/lib/jenkins/workspace/proton/label/rhel7-64/build && /usr/bin/cmake > -E cmake_depends "Unix Makefiles" > /var/lib/jenkins/workspace/proton/label/rhel7-64 > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python > /var/lib/jenkins/workspace/proton/label/rhel7-64/build > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python > > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python/CMakeFiles/docs-py.dir/DependInfo.cmake > --color= > Scanning dependencies of target docs-py > make[3]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make -f proton-c/bindings/python/CMakeFiles/docs-py.dir/build.make > proton-c/bindings/python/CMakeFiles/docs-py.dir/build > make[3]: Entering directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > cd > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python > && /usr/bin/python > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python/../../env.py > -- > PYTHONPATH=/var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python:/var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python > /usr/bin/epydoc -v --no-private --html -o > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python/html > > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python/proton.py > Error: Could not find a file or object named /var/lib/jenkins/ > workspace/proton/label/rhel7-64/proton-c/bindings/python/ > proton.py > Error: Nothing left to document! > make[3]: *** [proton-c/bindings/python/CMakeFiles/docs-py] Error 1 > make[3]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make[2]: *** [proton-c/bindings/python/CMakeFiles/docs-py.dir/all] Error 2 > make[2]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make[1]: *** [proton-c/CMakeFiles/docs.dir/rule] Error 2 > make[1]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make: *** [docs] Error 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-779) Building Proton documentation fails due to missing proton.py file
[ https://issues.apache.org/jira/browse/PROTON-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-779: Summary: Building Proton documentation fails due to missing proton.py file (was: RHEL 7 Build Issue) > Building Proton documentation fails due to missing proton.py file > - > > Key: PROTON-779 > URL: https://issues.apache.org/jira/browse/PROTON-779 > Project: Qpid Proton > Issue Type: Bug > Components: python-binding >Affects Versions: 0.9 > Environment: RHEL 7 >Reporter: Irina Boverman > Assignee: Darryl L. Pierce > > cmake -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo > -DCMAKE_SHARED_LINKER_FLAGS="-Wl,-z,relro" -DCMAKE_INSTALL_PREFIX=/usr > -DSYSINSTALL_BINDINGS=ON -DBUILD_RUBY=OFF .. > make all docs > Results in this: > Built target docs-c > make -f proton-c/bindings/python/CMakeFiles/docs-py.dir/build.make > proton-c/bindings/python/CMakeFiles/docs-py.dir/depend > make[3]: Entering directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > cd /var/lib/jenkins/workspace/proton/label/rhel7-64/build && /usr/bin/cmake > -E cmake_depends "Unix Makefiles" > /var/lib/jenkins/workspace/proton/label/rhel7-64 > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python > /var/lib/jenkins/workspace/proton/label/rhel7-64/build > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python > > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python/CMakeFiles/docs-py.dir/DependInfo.cmake > --color= > Scanning dependencies of target docs-py > make[3]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make -f proton-c/bindings/python/CMakeFiles/docs-py.dir/build.make > proton-c/bindings/python/CMakeFiles/docs-py.dir/build > make[3]: Entering directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > cd > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python > && /usr/bin/python > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python/../../env.py > -- > PYTHONPATH=/var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python:/var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python > /usr/bin/epydoc -v --no-private --html -o > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python/html > > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python/proton.py > Error: Could not find a file or object named /var/lib/jenkins/ > workspace/proton/label/rhel7-64/proton-c/bindings/python/ > proton.py > Error: Nothing left to document! > make[3]: *** [proton-c/bindings/python/CMakeFiles/docs-py] Error 1 > make[3]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make[2]: *** [proton-c/bindings/python/CMakeFiles/docs-py.dir/all] Error 2 > make[2]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make[1]: *** [proton-c/CMakeFiles/docs.dir/rule] Error 2 > make[1]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make: *** [docs] Error 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-779) RHEL 7 Build Issue
[ https://issues.apache.org/jira/browse/PROTON-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce reassigned PROTON-779: --- Assignee: Darryl L. Pierce > RHEL 7 Build Issue > -- > > Key: PROTON-779 > URL: https://issues.apache.org/jira/browse/PROTON-779 > Project: Qpid Proton > Issue Type: Bug > Components: python-binding >Affects Versions: 0.9 > Environment: RHEL 7 >Reporter: Irina Boverman > Assignee: Darryl L. Pierce > > cmake -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo > -DCMAKE_SHARED_LINKER_FLAGS="-Wl,-z,relro" -DCMAKE_INSTALL_PREFIX=/usr > -DSYSINSTALL_BINDINGS=ON -DBUILD_RUBY=OFF .. > make all docs > Results in this: > Built target docs-c > make -f proton-c/bindings/python/CMakeFiles/docs-py.dir/build.make > proton-c/bindings/python/CMakeFiles/docs-py.dir/depend > make[3]: Entering directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > cd /var/lib/jenkins/workspace/proton/label/rhel7-64/build && /usr/bin/cmake > -E cmake_depends "Unix Makefiles" > /var/lib/jenkins/workspace/proton/label/rhel7-64 > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python > /var/lib/jenkins/workspace/proton/label/rhel7-64/build > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python > > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python/CMakeFiles/docs-py.dir/DependInfo.cmake > --color= > Scanning dependencies of target docs-py > make[3]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make -f proton-c/bindings/python/CMakeFiles/docs-py.dir/build.make > proton-c/bindings/python/CMakeFiles/docs-py.dir/build > make[3]: Entering directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > cd > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python > && /usr/bin/python > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python/../../env.py > -- > PYTHONPATH=/var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python:/var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python > /usr/bin/epydoc -v --no-private --html -o > /var/lib/jenkins/workspace/proton/label/rhel7-64/build/proton-c/bindings/python/html > > /var/lib/jenkins/workspace/proton/label/rhel7-64/proton-c/bindings/python/proton.py > Error: Could not find a file or object named /var/lib/jenkins/ > workspace/proton/label/rhel7-64/proton-c/bindings/python/ > proton.py > Error: Nothing left to document! > make[3]: *** [proton-c/bindings/python/CMakeFiles/docs-py] Error 1 > make[3]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make[2]: *** [proton-c/bindings/python/CMakeFiles/docs-py.dir/all] Error 2 > make[2]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make[1]: *** [proton-c/CMakeFiles/docs.dir/rule] Error 2 > make[1]: Leaving directory > `/var/lib/jenkins/workspace/proton/label/rhel7-64/build' > make: *** [docs] Error 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Welcome Dominic Evans as a Qpid committer
On Mon, Dec 15, 2014 at 11:03:44AM +, Robbie Gemmell wrote: > The Apache Qpid PMC have voted to grant commit rights to Dominic Evans in > recognition of his contributions to and involvement with Proton. > > Welcome, Dominic! Congrats, Dominic! -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpPbsdn3MojE.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Fri, Dec 12, 2014 at 02:37:53PM -0500, Rafael Schloming wrote: > So what was the right thing to do here? I have to admit I struggled a bit > with the git portion of getting the whole thing landed. The changes were > (mostly) isolated to a subdirectory, yet whatever git incantations I seemed > to come up with forced me to painstakingly hand merge lots of conflicts > over and over again. What I ended up doing was mostly stumbled into in an > attempt to avoid all that grunge. Yeah, that's because there were several spots where master was merged into the javascript branch. So all of the changes that lived in master were already present in javascript (as a part of those merges) but were not in the same deltas as they live on master. So git was seeing the same changes show up in multiple separate commits and needed someone else to decide which was the right one. That's why rebasing your task branch in git is the Right Way(tm) to prepare a task branch for merging up to master. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpQwi65tvZDY.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Fri, Dec 12, 2014 at 02:15:21PM -0500, Andrew Stitcher wrote: > On Thu, 2014-12-11 at 17:11 -0500, Darryl L. Pierce wrote: > > Additionally, while working on a task branch, to resynch with master do > > a rebase: > > > > $ rebase -i master > > > > rather than merging master down onto your task branch. I saw a *lot* of > > that while examining the merge commits. Rebasing is by far one of the > > most awesome features of git. > > I agree and disagree with this simplistic position. > > If (and only if) your task branch is yours and yours alone and no one > has ever relied on it then you can safely rebase it. Actually I find > that rebasing is a lot more useful to get my commits in a logical > sequence of smaller working commits (by using rebase -i). Sorry, I should have clarified that this is my assumption, that a task branch is a private branch that you are working on and not one you're sharing with others. > On the other hand merging topic branches into master is also perfectly > sensible to finish off work on a topic branch especially a long lived > one. > > A long lived topic branch will necessarily have some level of merging > from master to keep it up to date. > > On the other hand I agree that merging from master just before merging > to master is irritating and pointless. > > Andrew > > -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpXSHYo3efcH.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Fri, Dec 12, 2014 at 09:48:05AM -0500, Clebert Suconic wrote: > In my experience there is no way to proper work with git without PRs and peer > review.. but that’s up to you guys. I feel like my comment, which was just a joke, might have been taken the wrong way. For that I apologize... -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpdG50SwZL1q.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Fri, Dec 12, 2014 at 01:43:45PM +, Gordon Sim wrote: > On 12/12/2014 12:16 PM, Darryl L. Pierce wrote: > >I like the idea of pull requests and explicit peer reviews for changes. > >But it's above my pay grade to do anything more than envy such a work > >flow. :D > > Pay grade isn't relevant on an Apache project. I was being facetious. I had mentioned pull requests a while back and got the response that (at least then) it wasn't being considered. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpfW3_aoy4LV.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Thu, Dec 11, 2014 at 06:47:06PM -0500, Clebert Suconic wrote: > We do a slightly different approach. If you guys are doing git I really think > you guys should consider it. > > 1. create a task branch : git checkout -b my-working-branch > > 2. create that voodoo that you do > > 3. when done, rebase your task branch on master : git rebase -i master > > 4. push to your github fork > 5. send a github Pull Request towards the github fork of the apache project > > 6. Someone else (yeah… someone who was not the original committer) should > merge it after review > > 7 That someone else will do: git merge --no-ff github/pr/PRNumber > ( We ellected to have merge committs with our committs, so we always have two > people to blame in case things go wrong) > > If you elect to not have the merge commit like we have, you can simply use > git pull.. and that should presever the Hashes from the original commit. > > 8. The apache bot will close the PR unless you rebased the commit (which you > are not supposed to) > > > > Why we do that? > > > I - to avoid merge errors like it just happened > II - it increases community > iii - we have a theory that everyone will do stupid things eventually... this > is an extra layer of protection :) > > > You could look at our README for more information: We are still getting > started with it and we have based this on how other apache projects are using > git and github: > > > https://github.com/apache/activemq-6/blob/master/README.md I like the idea of pull requests and explicit peer reviews for changes. But it's above my pay grade to do anything more than envy such a work flow. :D -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpVnaw8sVRxg.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Thu, Dec 11, 2014 at 07:19:13PM -0500, Clebert Suconic wrote: > Right but you can’t ever push -f on an apache branch. you can rebase as much > as you like .. and it’s awesome I agree… > But others would lose reference if you rebased and pushed -f.. that’s why > it’s forbidden at the apache git. > > And that’s why we are very careful on merging commits. I agree, as it can cause issues for someone who wasn't expecting history to have been fixed. Way back in the day, on a previous project, we had a commit hook that would reject merge commits, forcing the developer to fix their history before pushing anything. Does Apache support such hooks in projects? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp_XV0TUjtq5.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Thu, Dec 11, 2014 at 04:16:29PM -0500, Clebert Suconic wrote: > Rebasing and pushing is not a good option IMO. We have been using pull > requests from GitHub and pushing them through Apache. It's working very well > for us. > > Committing directly to Apachea may get you these issues. > > We can provide you guys more information on how we are doing on activemq6 if > you are interested. Additionally, while working on a task branch, to resynch with master do a rebase: $ rebase -i master rather than merging master down onto your task branch. I saw a *lot* of that while examining the merge commits. Rebasing is by far one of the most awesome features of git. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpeYfzyU3P0t.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Thu, Dec 11, 2014 at 04:16:29PM -0500, Clebert Suconic wrote: > Rebasing and pushing is not a good option IMO. We have been using pull > requests from GitHub and pushing them through Apache. It's working very well > for us. > > Committing directly to Apachea may get you these issues. > > We can provide you guys more information on how we are doing on activemq6 if > you are interested. So if we don't want to go back and fix history (which is fine) can we take steps to avoid pushing merge commits in future? It's just an ugly and confusing situation to get into with them. For anybody who's not familiar with a good git workflow: 1. create a task branch : git checkout -b my-working-branch 2. create that voodoo that you do 3. when done, rebase your task branch on master : git rebase -i master 4. checkout master and merge : git checkout master; git merge my-working-branch 5. make sure master is up to date : git pull --rebase 6. push your changes 7. profit! This way we don't have those ugly diamonds in the gitg graph and issues like this in future. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpHx0DDTzxNY.pgp Description: PGP signature
Re: Fixing the merge commit in the git repo?
On Thu, Dec 11, 2014 at 02:35:05PM -0500, Rafael Schloming wrote: > Can you provide a bit more detail? I'm not the most expert git user, so I'm > not sure exactly what you're asking for, much less how to do it. In my case, my usual workflow is to do small changes and commit them locally. If a change goes with a commit a few back I'll do: $ git commit -a --fixup=[early commit's hash] Then, when I'm ready to commit I'll do: $ git rebase -i HEAD~[# of commits to process] --autosquash and git will put the commits in the right order so I can squash them down. Normally I just do "HEAD~25" rather than counting the commits since we usually had a nice, linear stream of commits in our old repo that was cloned from Subversion. But when I did it this time that the commits overlapped with this merge commit: ===[snip]=== commit e8029597b825e49ee5ab2b7c8bd5d042e57321bb Merge: db437cf a5cb27e Author: Rafael Schloming Date: Fri Nov 28 08:43:03 2014 -0500 Merge branch 'javascript' ===[snip]=== This made git them try to rebase all of those changes, which numbered up to about 60 or so commits that are out of sequence. So I had to abort that rebase and count the commits instead and do it again. When merging branches, etc. we can avoid this if, before merging branch A into master we go to branch A and rebase it on master. That way we avoid merge commits like this. To fix it, you would need to: 1. git rebase -i HEAD~[# of commits to the merge above] 2. when it pops up the list of commits, just save (you're not going to re-order them at that point) 3. check for any merge commits, dual adds or deletes with "git status" 4. fix any issues that you see: fix merges, etc. 5. git add (or del) files as needed 6. git rebase --continue 7. go to 3 as needed until all is fixed -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpGCgaCe_wBM.pgp Description: PGP signature
[jira] [Updated] (PROTON-756) add a new/simpler setup.py for python bindings
[ https://issues.apache.org/jira/browse/PROTON-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-756: Attachment: 0001-PROTON-756-Add-the-proton-directory-to-the-install-p.patch Appends "proton" the Python install path for the bindings. > add a new/simpler setup.py for python bindings > -- > > Key: PROTON-756 > URL: https://issues.apache.org/jira/browse/PROTON-756 > Project: Qpid Proton > Issue Type: Bug > Components: python-binding >Reporter: Rafael H. Schloming >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.9 > > Attachments: > 0001-PROTON-756-Add-the-proton-directory-to-the-install-p.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Fixing the merge commit in the git repo?
Would it be possible for someone (Rafi?) to fix the merge commits in the Git repo? I'm working on some stuff and, when I tried to do a rebase I accidentally went a few commits further back and git wanted to then rebase 65 commits... On our individual ends, once it's done, we should be able to just do: $ git pull --rebase and have our individual repos fixed as a result. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpsO2YbjQg7u.pgp Description: PGP signature
[jira] [Resolved] (PROTON-765) 64-bit values are not being properly marshalled in Ruby on 32-bit systems
[ https://issues.apache.org/jira/browse/PROTON-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce resolved PROTON-765. - Resolution: Fixed Fix Version/s: 0.9 > 64-bit values are not being properly marshalled in Ruby on 32-bit systems > - > > Key: PROTON-765 > URL: https://issues.apache.org/jira/browse/PROTON-765 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding >Affects Versions: 0.8 > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.9 > > > On 32-bit Linux using both Ruby 1.8 and 2.0 I see the following when setting > a Bignum value: > 3: 2) A data object can hold a decimal64 > 3: Failure/Error: @data.decimal64 = value > 3: RangeError: > 3:bignum too big to convert into `unsigned long' > 3: # ./lib/qpid_proton/data.rb:633:in `pn_data_put_decimal64' > 3: # ./lib/qpid_proton/data.rb:633:in `decimal64=' > 3: # ./spec/qpid/proton/data_spec.rb:318:in `block (2 levels) in > ' > It seems that a >32-bit value can't be held on a 32-bit system in Proton. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-768) DECIMAL32 values are not consistent in Ruby
Darryl L. Pierce created PROTON-768: --- Summary: DECIMAL32 values are not consistent in Ruby Key: PROTON-768 URL: https://issues.apache.org/jira/browse/PROTON-768 Project: Qpid Proton Issue Type: Bug Components: ruby-binding Reporter: Darryl L. Pierce Assignee: Darryl L. Pierce On 32-bit EL6 running Ruby 1.8.7, DECIMAL32 values change. For example, the following happened during running the rspec tests: 2) A data object can hold a decimal32 Failure/Error: expect(@data.decimal32).to eq(value) expected: 1161181977 got: 1536795250 (compared using ==) # ./spec/qpid/proton/data_spec.rb:304 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-755) Update Ruby unit tests to use version 4.7 of Minitest
[ https://issues.apache.org/jira/browse/PROTON-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated PROTON-755: Summary: Update Ruby unit tests to use version 4.7 of Minitest (was: Update Ruby unit tests to use Minitest) > Update Ruby unit tests to use version 4.7 of Minitest > - > > Key: PROTON-755 > URL: https://issues.apache.org/jira/browse/PROTON-755 > Project: Qpid Proton > Issue Type: Bug > Components: ruby-binding > Reporter: Darryl L. Pierce > Assignee: Darryl L. Pierce > Fix For: 0.9 > > > The old Test::Unit module names have been collapsed down to Minitest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
