RE: [Zope] Large XML files
"Parsed XML" (the successor to XML Document) should handle large documents less expensively. No code has been released yet, but something should be available soon (in a couple of weeks?)... Keep an eye on http://www.zope.org/Wikis/DevSite/Projects/ParsedXML and you may want to subscribe to the zope-xml mailing list. --- "Sardonic Slaughterer of Sacred Cows" Todd Coram - mailto:[EMAIL PROTECTED] Digital Creations, Inc - http://www.digicool.com Publishers of Zope - http://www.zope.org -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Phil Harris Sent: Tuesday, December 05, 2000 6:03 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [Zope] Large XML files Paul, The XMLDocument type is rather 'expensive' and you may be able to 'get away' with using a simpler type such as DTMLDocument. If you don't need the DOMification of your XML when you put it into Zope this is something i'd look at.. I have done this with large documents and then used an external method to process them with XSLT, XPath etc. hth Phil [EMAIL PROTECTED] - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, December 05, 2000 10:01 PM Subject: [Zope] Large XML files I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this? Thanks, P. Johnson ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Large XML files
On Tue, 5 Dec 2000 23:02:38 -, "Phil Harris" [EMAIL PROTECTED] wrote: The XMLDocument type is rather 'expensive' and you may be able to 'get away' with using a simpler type such as DTMLDocument. Simpler, but thats not necessarily an advantage. DTMLDocument will store the whole document in memory, but XMLDocument uses ZODB effectively so that it only loads the DOM nodes that are in use. If you are always using the *whole* document then this is no help, and simpler may indeed be better. From: [EMAIL PROTECTED] I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this? For a machine of that size I suggest you use only one publisher thread (thats -T 1 on the command line), rather than the default of 4. Each thread gets a copy of the ZODB object cache, and you probably dont want to keep four copies of your 700k document in memory. Toby Dickenson [EMAIL PROTECTED] ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Large XML files
Definetly consider a stripped down linux install for this machine instead. just make sure almost nothing else is running as well (A lots ditros start all sorts of stuff up for you). Edward Muller - [EMAIL PROTECTED] - [EMAIL PROTECTED] Grow a ponytail -- view it as your telepathic antenna to other Linux Kernel Developers. -- Jeff V. Merkey On Tue, 5 Dec 2000, Jim Hebert wrote: You could check into the ExtFile product (search on zope.org) and see if that makes a perf difference. That said, just about anything on that setup is going to be slow!! Please excuse the os advocacy, but given that this is likely all you'd dream of running on this server anyways (ie it's not like it's also going to be your Exchange server), you might consider putting something that will leave more spare RAM and cpu time for Zope on that box, e.g. Linux or some other OS more suited towards the low end server hardware market. Or buy a nicer box to run this on. If you don't have existing unix skills in-house then throwing money at hardware might be less expensive than investing in human resources to get a Linux box going. (Whew, being even-handed is so much harder than being an OS bigot...;-) Sorry that's probably not what you wanted to hear. =) jim On Tue, 5 Dec 2000 [EMAIL PROTECTED] wrote: I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this? Thanks, P. Johnson ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Large XML files
You could check into the ExtFile product (search on zope.org) and see if that makes a perf difference. That said, just about anything on that setup is going to be slow!! Please excuse the os advocacy, but given that this is likely all you'd dream of running on this server anyways (ie it's not like it's also going to be your Exchange server), you might consider putting something that will leave more spare RAM and cpu time for Zope on that box, e.g. Linux or some other OS more suited towards the low end server hardware market. Or buy a nicer box to run this on. If you don't have existing unix skills in-house then throwing money at hardware might be less expensive than investing in human resources to get a Linux box going. (Whew, being even-handed is so much harder than being an OS bigot...;-) Sorry that's probably not what you wanted to hear. =) jim On Tue, 5 Dec 2000 [EMAIL PROTECTED] wrote: I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this? Thanks, P. Johnson ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Large XML files
Paul, The XMLDocument type is rather 'expensive' and you may be able to 'get away' with using a simpler type such as DTMLDocument. If you don't need the DOMification of your XML when you put it into Zope this is something i'd look at.. I have done this with large documents and then used an external method to process them with XSLT, XPath etc. hth Phil [EMAIL PROTECTED] - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, December 05, 2000 10:01 PM Subject: [Zope] Large XML files I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this? Thanks, P. Johnson ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
