Re: [libxml-devel] libxml crash

2008-01-11 Thread Sean Chittenden
> I have yet to have valgrind produce anything meaningful in debugging > this environment. I have tried the suppressions, but alas, nothing. If > someone can offer an effective way of using it, I am quite interested. The best debugging mechanism that I've found to date is to do something along t

Re: [libxml-devel] libxml crash

2008-01-11 Thread Dan Janowski
Marc, I have yet to have valgrind produce anything meaningful in debugging this environment. I have tried the suppressions, but alas, nothing. If someone can offer an effective way of using it, I am quite interested. As for this bug, the context code is pretty simple. This is different than

Re: [libxml-devel] libxml crash

2008-01-11 Thread Marc Munro
My last experience with ruby and valgrind was kinda unpleasant. Maybe with ruby 1.9 it will be better but I don't really have the stomach for changing my entire test environment right now. Worst still, as soon as I had packed up a tar file of the code that lead to the crash, it stopped crashing.

Re: [libxml-devel] libxml crash

2008-01-11 Thread Todd Fisher
On 1/10/08, Marc Munro <[EMAIL PROTECTED]> wrote: > This is with libxml and libxsl built from SVN 3 days ago You might try running with valgrind. > > If I run with gdb, my program runs normally. > > Any suggestions on how I can further track this down? > > __ > Marc > > > *** glibc detected *** r

[libxml-devel] libxml crash

2008-01-10 Thread Marc Munro
This is with libxml and libxsl built from SVN 3 days ago. If I run with gdb, my program runs normally. Any suggestions on how I can further track this down? __ Marc *** glibc detected *** ruby: corrupted double-linked list: 0x0914ed28 *** === Backtrace: = /lib/libc.so.6[0xb7cf7169]

Re: [libxml-devel] libxml crash

2007-08-25 Thread Sean Chittenden
> If things are being freed unintentionally, then there are not enough > ruby to ruby cross-references to maintain integrity during mark and > sweep. I actually think this could be done, but an external hash table of ptr's of both ruby and libxml obj's must be setup/maintained and it's problem

Re: [libxml-devel] libxml crash

2007-08-25 Thread Charlie Savage
> A single ruby object holding on to an xmlNode should inhibit collection of an entire tree. Individual ruby xml nodes can be reaped, but there should be no xmlFree done by anything but the document level object. As for an xmlNode that is not associated with a document, it must call xmlFree

Re: [libxml-devel] libxml crash

2007-08-25 Thread Ross Bamford
On Fri, 24 Aug 2007 16:47:58 +0100, Dan Janowski <[EMAIL PROTECTED]> wrote: > If things are being freed unintentionally, then there are not enough > ruby to ruby cross-references to maintain integrity during mark and > sweep. A single ruby object holding on to an xmlNode should inhibit > collectio

Re: [libxml-devel] libxml crash

2007-08-24 Thread Dan Janowski
My analysis leads me to a similar conclusion. Reference counting is out and there is no need for it. This is my hypothesis: If things are being freed unintentionally, then there are not enough ruby to ruby cross-references to maintain integrity during mark and sweep. A single ruby object hol

Re: [libxml-devel] libxml crash

2007-08-23 Thread Tom Hughes
In message <[EMAIL PROTECTED]> Marc Munro <[EMAIL PROTECTED]> wrote: > On Thu, 2007-23-08 at 12:14 -0600, Charlie Savage wrote: > > > If not, can you install a callback into libxml to be alerted when a node > > is freed? If so, then you could decrement the reference count. But > > you

Re: [libxml-devel] libxml crash

2007-08-23 Thread Sean Chittenden
> For SWIG I implemented support for mapping between C pointers and > Ruby objects by using a Hash Table (ruby's built in one, > st_table). That allows you to enforce a one-to-one Ruby to C > object mapping. Do we want to do something like that here? It doesn't exist now and I'm inclined

Re: [libxml-devel] libxml crash

2007-08-23 Thread Sean Chittenden
> My patches (to which I see you have already been pointed) are quite > likely to help with this. They're only a band aid really though - what > is really needed is a ground up review/redesign of the way the two > object models are tied together as the current scheme basically > doesn't work at all

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
So, when can you have it ready ;-) Hah. Let's what other people think - I could be way off base since I haven't looked in-depth at the bindings code. Thoughts everyone? If this idea has merit, then the trickiest bit would be making sure there is a one-to-one Ruby document object to libxml

Re: [libxml-devel] libxml crash

2007-08-23 Thread Marc Munro
On Thu, 2007-23-08 at 13:48 -0600, Charlie Savage wrote: > I don't think your Ruby code would change at all. Under the covers, the > bindings would see that you are adding root to to doc, and so the Ruby > object pointing to root would give up ownership. > > You'd then setup a mark function fo

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
I'm not sure why this seems tough. If we have a reference to a leaf when we want to garbage collect the rest of the tree, I am suggesting removing that leaf from the tree prior to freeing it. You don't view the tree as atomic? Seems to me it is. When you add the Ruby objects to a tree, the Ru

Re: [libxml-devel] libxml crash

2007-08-23 Thread Marc Munro
On Thu, 2007-23-08 at 13:04 -0600, Charlie Savage wrote: > > > Instead of trying to ensure that we free the ruby > > object when the xmlNode is freed, we have to ensure that such nodes > > referenced by ruby objects are not freed at all by xmlFree. > > Boy - that's tough in a tree. What if you

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
I don't know if libxml2 provides such callbacks, but I don't see how this would work anyway. The xmlNode has no idea what ruby objects are referencing it. Right - you have to add in mapping between Ruby objects and libxml objects (its a hash table keyed on the C pointer having pointers to the

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
Simple - you create an element object in ruby, then add that element as the child of another element. The ruby element object still exists and still points at the same element though (and can be used to make changes to that element in the tree). Is this avoidable by unsetting the Ruby object's f

Re: [libxml-devel] libxml crash

2007-08-23 Thread Tom Hughes
In message <[EMAIL PROTECTED]> Charlie Savage <[EMAIL PROTECTED]> wrote: > > The problems that I have discovered stem from libxml freeing entire > > document trees while there are still ruby objects referencing the nodes. > > When those ruby objects are subsequently garbage-collected, th

Re: [libxml-devel] libxml crash

2007-08-23 Thread Tom Hughes
In message <[EMAIL PROTECTED]> Marc Munro <[EMAIL PROTECTED]> wrote: > On Thu, 2007-23-08 at 11:13 -0600, Charlie Savage wrote: > > Ah - interesting. A hybrid approach is extremely difficult because it > > means you have to keep track of the ruby to libxml object mapping. For > > examp

Re: [libxml-devel] libxml crash

2007-08-23 Thread Tom Hughes
In message <[EMAIL PROTECTED]> Charlie Savage <[EMAIL PROTECTED]> wrote: > > The problem is that some nodes are allocated by libxml by parsing > > documents, processing xsl, etc, and some directly by ruby. I think the > > hybrid approach is necessary in order to get the speed benefits o

Re: [libxml-devel] libxml crash

2007-08-23 Thread Marc Munro
Dang! And I really intended to butt out of this discussion :-) On Thu, 2007-23-08 at 12:14 -0600, Charlie Savage wrote: > > The problems that I have discovered stem from libxml freeing entire > > document trees while there are still ruby objects referencing the nodes. > > When those ruby objects

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
The problems that I have discovered stem from libxml freeing entire document trees while there are still ruby objects referencing the nodes. When those ruby objects are subsequently garbage-collected, the xmlNodes in those objects have already been freed and sadness ensues. But I think the curren

Re: [libxml-devel] libxml crash

2007-08-23 Thread Marc Munro
On Thu, 2007-23-08 at 11:13 -0600, Charlie Savage wrote: > Ah - interesting. A hybrid approach is extremely difficult because it > means you have to keep track of the ruby to libxml object mapping. For > example, say you have some ruby code that create a libxml node. That > ruby object would

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
The problem is that some nodes are allocated by libxml by parsing documents, processing xsl, etc, and some directly by ruby. I think the hybrid approach is necessary in order to get the speed benefits of using libxml, and I think a hybrid approach to freeing is therefore necessary. It does seem t

Re: [libxml-devel] libxml crash

2007-08-23 Thread Marc Munro
On Thu, 2007-23-08 at 10:45 -0600, Charlie Savage wrote: > Would the solution be a simple as having ruby nodes *never* freeing > libxml objects except the toplevel document (since it sounds like libxml > cleans up after itself). Not all nodes wind up in documents, so unless you're prepared for

Re: [libxml-devel] libxml crash

2007-08-23 Thread Charlie Savage
Tom, Your crash is a typical one - it looks like it is caused by ruby still having a handle to a libxml object that has been attached to another libxml object that has then been freed. Would the solution be a simple as having ruby nodes *never* freeing libxml objects except the toplevel docum

Re: [libxml-devel] libxml crash

2007-08-23 Thread Marc Munro
On Thu, 2007-23-08 at 07:33 -0400, Doug Bryant wrote: > This is most likely related to a copy node bug which has been fixed in > a .4 release. Ross fixed per an issue I opened. Although the .4 > release is labled as a pre-release, it is stable with the exception of > the memory bug. > > Upgrade

Re: [libxml-devel] libxml crash

2007-08-23 Thread Doug Bryant
This is most likely related to a copy node bug which has been fixed in a .4 release. Ross fixed per an issue I opened. Although the .4 release is labled as a pre-release, it is stable with the exception of the memory bug. Upgrade and it will most likely fix the problem. See issue #4635 (now clo

Re: [libxml-devel] libxml crash

2007-08-23 Thread Tom Hughes
In message <[EMAIL PROTECTED]> Marc Munro <[EMAIL PROTECTED]> wrote: > I have a reproducible crash using libxml. I'd like some suggestions on > how I can track this down as it's become a show-stopper for my current > project. This is from a libxml gem created from svn this morning. Your

Re: [libxml-devel] libxml crash

2007-08-22 Thread Marc Munro
On Wed, 2007-22-08 at 18:21 -0600, Charlie Savage wrote: > > Did you try these patches? > > http://rubyforge.org/pipermail/libxml-devel/2007-June/000314.html Hmmm. I can apply the leak2 patch but not the leak1 patch which seems way more significant: marc:[libxml]$ patch -p1 signature.asc Des

Re: [libxml-devel] libxml crash

2007-08-22 Thread Marc Munro
On Wed, 2007-22-08 at 18:21 -0600, Charlie Savage wrote: > Marc, > > > I have a reproducible crash using libxml... > Can you provide the test data? That would be hugely helpful. I could but it's a bit of a monster, probably timing critical, and likely to run differently on different platforms.

Re: [libxml-devel] libxml crash

2007-08-22 Thread Charlie Savage
Marc, I have a reproducible crash using libxml. I'd like some suggestions on how I can track this down as it's become a show-stopper for my current project. This is from a libxml gem created from svn this morning. Can you provide the test data? That would be hugely helpful. Also what OS, p

[libxml-devel] libxml crash

2007-08-22 Thread Marc Munro
Folks, I have a reproducible crash using libxml. I'd like some suggestions on how I can track this down as it's become a show-stopper for my current project. This is from a libxml gem created from svn this morning. I can write and debug C but I'd like some suggestions on what to look for, and wh