Well, what I'm getting at is that many scripting languages may
guarantee (or provide a way to guarantee) thread safety of the
reference counting operations. In Perl, if you want access to a
variable from multiple threads, you explicitly have to designate it as
"shared" so that the variable can be read and written safely. Python
has a "global interpreter lock" that a Python/C++ wrapper would
presumably use. I'm not sure about Ruby et al.
The performance of C++/PB in Perl and Python is obviously going to be
vastly better than that of a pure-Perl or pure-Python implementation,
even with this deep-copying going on, and I really doubt it would be a
bottleneck in any actual Perl or Python application (I mean, if you
really want the speed, those aren't the languages to use...). But it
would still be nice to have a way to unlock the maximum speed. Let me
know if you change your mind!
-dave
On Dec 3, 3:31 pm, Kenton Varda <[EMAIL PROTECTED]> wrote:
> It still adds a lot of complication. And I think most cases where people
> start out thinking thread-safety won't be an issue, particularly with
> reference counting, they later find out otherwise.
>
> On Wed, Dec 3, 2008 at 3:22 PM, Dave Bailey <[EMAIL PROTECTED]> wrote:
>
> > What if thread safety wasn't an issue?
>
> > -dave
>
> > On Dec 3, 2:41 pm, Kenton Varda <[EMAIL PROTECTED]> wrote:
> > > Ehhh... Reference counting is slow (assuming it needs to be
> > thread-safe),
> > > and I think even adding it as an option would add an excessive amount of
> > > complication to the system.
>
> > > On Wed, Dec 3, 2008 at 2:04 PM, Dave Bailey <[EMAIL PROTECTED]> wrote:
>
> > > > On Dec 3, 2:00 pm, Dave Bailey <[EMAIL PROTECTED]> wrote:
> > > > > On Dec 2, 10:49 pm, Kenton Varda <[EMAIL PROTECTED]> wrote:
>
> > > > > > C++ compatibility matters because eventually we want to be able to
> > > > generate
> > > > > > Python code which just wraps C++ code for efficiency. C++ isn't
> > > > garbage
> > > > > > collected, so append() can't easily be implemented in this case
> > without
> > > > > > having ownership problems. Slice assignment has the same problem.
> > > > > > Also note that even pure-python protocol buffers have a sort of
> > > > "ownership"
> > > > > > issue: Sub-messages all contain pointers back to their parents, so
> > > > that
> > > > > > when a sub-message is modified, the parent's cached size can be
> > marked
> > > > > > dirty. (Also, singular sub-messages have to inform their parents
> > when
> > > > the
> > > > > > first field within them is set, but that doesn't apply here.)
>
> > > > (Here is my post without all of the ridiculous formatting):
>
> > > > While you're on this topic, I ran into this ownership issue while
> > > > implementing the Perl/XS wrapper around the generated C++ code. I
> > > > think it is the same issue that would face the author of a Python or
> > > > Ruby C++ extension of the generated C++. I ended up having to new() a
> > > > copy of every message that I transferred from C++ to Perl or vice
> > > > versa. So, for example, a statement like
>
> > > > $team->member($i)->set_first_name('Dave');
>
> > > > won't have the same effect as (C++)
>
> > > > team.mutable_member(i)->set_first_name("Dave");
>
> > > > because $team->member($i) will generate a copy of the underlying C++
> > > > object, so that it can be managed by Perl's reference counting without
> > > > any concern as to whether or not the underlying C++ object has been
> > > > deleted because the containing message went out of scope.
>
> > > > Anyway, I thought it might be possible to allow for shared ownership
> > > > of a message object if there were a reference counted variant of
> > > > RepeatedPtrField<T> (something like RepeatedSharedPtrField<T> or
> > > > whatever), which would provide incref() and decref() methods such that
> > > > Perl and C++ could use the same underlying C++ objects in the
> > > > generated code. This would really help the performance of the Perl/XS
> > > > code if all of that copy construction could be avoided somehow. The C+
> > > > + code generator would need an option that would instruct it to
> > > > generate RepeatedSharedPtrField<T> members (and incref and decref
> > > > calls, where appropriate) for repeated messages (instead of using the
> > > > default RepeatedPtrField<T>).
>
> > > > What do you think? Is something like this possible, even though it
> > > > would require a change to protobuf? It is an issue for all {Python,
> > > > Perl, Ruby, ...}/C++ extension wrappers for Protocol Buffers. I have
> > > > found that protobuf is a faster Perl data serialization mechanism that
> > > > the (generic) Storable module, but I think it can be even faster.
>
> > > > -dave
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---
