Re: All about proton memory management (or C++, python and Go - Oh My!)
On Wed, 2015-08-19 at 10:45 -0400, Ken Giusti wrote: Nicely done Alan! One point - I'm a little confused about your advice regarding pn_object_decref: The proton C API has standard reference counting rules (but see [1] below) * A pointer returned by a pn_ function is either borrowed by the caller, or the caller owns a reference (the API doc says which.) * The owner of a reference must call pn_object_decref() exactly once to release it. Really? What about those proton objects that have a pn_X_free() method? I believe the corresponding pn_X() allocation methods pass back an owning reference to the object. If a pn_X_free() exists (and the user never calls pn_object_incref() on the object), shouldn't the user use the pn_X_free() method instead? Maybe the doc should make that distinction, e.g.: * if an object of type pn_X_t has a corresponding pn_X_free() method, call the pn_X_free() method to release your reference to the object. * otherwise, if you have called pn_object_incref() on the object, you must call pn_object_decref() on the object to release your reference. From skimming the source, I believe pn_x_free looks at the refcount itself and behaves like pn_object_decref without the if (--refcount=0) pn_x_free(me) So in theory they *should* work together. The proton lib has no way of knowing whether the last thing to release an object will be an internal decref or a pn_x_free from the user, so I sure *hope* they work together. However I did want to make the point that you should EITHER use refcounts only OR free only, not both. Even if it works today your code or your head may explode later. IMO normal C code (including bindings to languages without finalizers like Go) should use free only, bindings to automated memory languages like python and soon C++ should use refcounts only. Of course the difference between theory and practice is that in theory they are the same and in practice they are different.
Re: All about proton memory management (or C++, python and Go - Oh My!)
On Tue, 2015-08-18 at 12:09 -0400, Andrew Stitcher wrote: On Tue, 2015-08-18 at 07:38 -0400, Rafael Schloming wrote: Nice writeup! I agree. Andrew [Did you make a pass through the doc to ensure that the claimed API doc is actually there? that is, doc on ownership and scope?] No, will add to my TODO list :) Some of it definitely is but I don't know if is uniform and complete. The existing doc does not refer to refcounts, it talks of pointers becoming invalid. I think that is the correct language - we don't want to give the impression that refcounts are mandatory. The discussion of refcounting can clarify that becomes invalid means the implied _reference_ becomes invalid. The actual object might not be freed immediately (with or without user refcounts because of containment) but in any case it is no longer your business. You must treat that pointer or the implied reference as invalid or you are deep in works in tests, core dumps in production territory. Thanks for the feedback! Alan.
Re: All about proton memory management (or C++, python and Go - Oh My!)
On Tue, 2015-08-18 at 07:38 -0400, Rafael Schloming wrote: Nice writeup! I agree. Andrew [Did you make a pass through the doc to ensure that the claimed API doc is actually there? that is, doc on ownership and scope?]
All about proton memory management (or C++, python and Go - Oh My!)
I've been doing a lot of thinking about memory management and proton bindings in 4 languages (C, Go, python and C++) and I have Seen The Light. Here is a write-up, I'd appreciate feedback in the form of email, reviewboard diffs, regular diffs, or just commit improvements if you're a commiter. I will add this to the official proton API doc after incorporating feedback: https://github.com/apache/qpid-proton/blob/master/docs/markdown/memory_ management.md