Re: Generating HTML (was Gump Threashing/Spinning)
> > > Now Gump generates it's xdocs using an object tree structure. Watching > the > > > python memory grow from 20M (after loading all XML) to 136M (during > > > generating these pages) it has some sort of leak (actual or effective) > > > > ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) > > instead of DOM to generate the object tree. > > I wonder if it is some sort of circular dependency (amonst the objects) so > when I destroy a tree (by pointing the variable to a new one) I wonder if it > truely gets destroyed. I know the DOM has an unlink() method for some good > reason, along these lines. > > There is so much thrown up into memory, more with translations to try to > cope with character sets (and binary junk) and such. I no longer believe > that any is being thrown away when I mean it to be... Adding an unlink() to the tree, and calling it, seems to keep memory usage down to 36M and not 136M. Seems Python needs a hand in recognizing when things are no longer used. BTW: It seems I've persuaded forrest to play happily again by removing the 'dependency path' (from cause to project) that I'd added. No clue why, but whatever. Gump ran on LSD last night, although it still took a long long time. However, we are closer to normal again & able to install on Apache hardware as a valid test. regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Generating HTML (was Gump Threashing/Spinning)
> > First, I like the dynamic 'tree of nodes' based approach to writing > > HTML/XML, rather than template > > I like merging the concepts. Once you've built the tree, flatten the > part of it that will make up the page, and feed that to the template > engine. Even if you don't use a template engine, seperate the flattening > and splitting from the transformation step. I am game to explore this. HTML or XML (xdoc) output ought be the same code, just with different templates, so we can have 'pure Python generating HTML' and/or 'forrest on xdocs' without us having to compromise, right? > > This is different from stuff like anakia, where the template walks the > tree directly (bad). I think this is the key. I'd really like to find a good way to go from a tree of objects (not DOM, our models and context, etc.) to variables for templates, or whatever. I don't want this to be kludgy, and feel that cheetah/Python likely have some sort of slick solution. BTW: I can see the need to use includes, and I can see 'if' directives, but maybe we'd still want to use bits of templates glued together to get re-use. I think it depends upon the tree to tempalte choice we make... > > Second, I didn't realize that Python DOM had nice serialization mechanism. > > Maybe I should've used that from the start. > > No idea what you're talking about :D The Python DOM tree will serialize to an XML stream (unlike old DOMs I'm used to). I wrote some stuff I should never have in Python. Hey, we are here to learn, right? > > Now Gump generates it's xdocs using an object tree structure. Watching the > > python memory grow from 20M (after loading all XML) to 136M (during > > generating these pages) it has some sort of leak (actual or effective) > > ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) > instead of DOM to generate the object tree. I wonder if it is some sort of circular dependency (amonst the objects) so when I destroy a tree (by pointing the variable to a new one) I wonder if it truely gets destroyed. I know the DOM has an unlink() method for some good reason, along these lines. There is so much thrown up into memory, more with translations to try to cope with character sets (and binary junk) and such. I no longer believe that any is being thrown away when I mean it to be... > > Fixing this area is clearly important, and I'd appreciate all insights... > > No insights here, just babbling along ;) Better than my just listening to the babbling in my head. ;-) regards Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Generating HTML (was Gump Threashing/Spinning)
Leo Simons wrote: Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective) ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) instead of DOM to generate the object tree. you are generating a dom on the flight! ahhh please, no matter what you do with forrest, please use a template engine instead of a DOM. there is no need for a dom for gump. -- Stefano, thinking seriously that it's time to take a look at the internals of gumpy smime.p7s Description: S/MIME Cryptographic Signature
Re: Generating HTML (was Gump Threashing/Spinning)
Adam Jack wrote: We should probably use a template engine. I'm sure there's a python equivalent for something like velocity (or smarty). First, I like the dynamic 'tree of nodes' based approach to writing HTML/XML, rather than template I like merging the concepts. Once you've built the tree, flatten the part of it that will make up the page, and feed that to the template engine. Even if you don't use a template engine, seperate the flattening and splitting from the transformation step. This is different from stuff like anakia, where the template walks the tree directly (bad). -- in the main because of pleasant experiences with the Perl modules for this in the past. That said, even though there are some huge fields here (build output logs) hehehe. Look here: http://www.cheetahtemplate.org/docs/users_guide_html_multipage/language.constructs.html this is what things like #include are for! maybe a template mechanism is enough for the simple structure (results tables, lists, etc.) that we have. We have huge pages, with options missing (or not) based off content. I'm not sure any template could cope with that, but I am open minded to it. The biggest ISP in the Netherlands with the biggest content portal in the Netherlands uses a template engine as a core part of their view layer (but don't tell 'em ;) :D. At least they used to... Second, I didn't realize that Python DOM had nice serialization mechanism. Maybe I should've used that from the start. No idea what you're talking about :D Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective) ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff) instead of DOM to generate the object tree. Fixing this area is clearly important, and I'd appreciate all insights... No insights here, just babbling along ;) -- cheers, - Leo Simons --- Weblog -- http://leosimons.com/ IoC Component Glue -- http://jicarilla.org/ Articles & Opinions -- http://articles.leosimons.com/ --- "We started off trying to set up a small anarchist community, but people wouldn't obey the rules." -- Alan Bennett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Generating HTML (was Gump Threashing/Spinning)
Perhaps we should run cocoon/fforrest live and just update the xdocs underneath? Adam Jack wrote: We should probably use a template engine. I'm sure there's a python equivalent for something like velocity (or smarty). First, I like the dynamic 'tree of nodes' based approach to writing HTML/XML, rather than template -- in the main because of pleasant experiences with the Perl modules for this in the past. That said, even though there are some huge fields here (build output logs) maybe a template mechanism is enough for the simple structure (results tables, lists, etc.) that we have. We have huge pages, with options missing (or not) based off content. I'm not sure any template could cope with that, but I am open minded to it. Second, I didn't realize that Python DOM had nice serialization mechanism. Maybe I should've used that from the start. Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective): Fixing this area is clearly important, and I'd appreciate all insights... regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Generating HTML (was Gump Threashing/Spinning)
> We should probably use a template engine. I'm sure there's a python > equivalent for something like velocity (or smarty). First, I like the dynamic 'tree of nodes' based approach to writing HTML/XML, rather than template -- in the main because of pleasant experiences with the Perl modules for this in the past. That said, even though there are some huge fields here (build output logs) maybe a template mechanism is enough for the simple structure (results tables, lists, etc.) that we have. We have huge pages, with options missing (or not) based off content. I'm not sure any template could cope with that, but I am open minded to it. Second, I didn't realize that Python DOM had nice serialization mechanism. Maybe I should've used that from the start. Now Gump generates it's xdocs using an object tree structure. Watching the python memory grow from 20M (after loading all XML) to 136M (during generating these pages) it has some sort of leak (actual or effective): Fixing this area is clearly important, and I'd appreciate all insights... regards, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
