[Zope] request for advice: using ZCatalog
hello all i'm about to jump into uncharted waters (well, for me at least) i have a news site, and would like to use ZCatalog to present, index, etc news. all's well and good. i have a functioning ZClass, that is catalog aware, and searchable. what i'd like to know is -the stability of ZCatalog as of 2.2.5. -what's in store in 2.3 -upgradability from 2.2.5 to 2.3 -heartstopping bugs that i may have not encountered yet -performance speed/response. with my limited tests[1], i got ok response time, compared to what i have now where i iterate through objectItems/Values. but when i cache with CachePool[2], response is better. correct me if i'm wrong. i'd like to not go thru what i'm going thru now. but will this hold water? especially since i expect more requests. thanks ---8 [1] /usr/sbin/ab -n 300 -c 200 myurl This is ApacheBench, Version 1.3c $Revision: 1.38 $ apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ Server Software:Apache/1.3.9 Server Hostname:203.106.2.200 Server Port:80 Document Path: myurl Document Length:9644 bytes Concurrency Level: 200 Time taken for tests: 33.586 seconds Complete requests: 300 Failed requests:0 Total transferred: 3030864 bytes HTML transferred: 2916288 bytes Requests per second:8.93 Transfer rate: 90.24 kb/s received Connnection Times (ms) min avg max Connect:145 162 Processing: 736 14748 25346 Total:737 14793 25508 [2] /usr/sbin/ab -n 300 -c 200 myotherurl This is ApacheBench, Version 1.3c $Revision: 1.38 $ apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ Server Software:Apache/1.3.9 Server Hostname:203.106.2.200 Server Port:80 Document Path: myotherurl Document Length:9524 bytes Concurrency Level: 200 Time taken for tests: 7.646 seconds Complete requests: 300 Failed requests:0 Total transferred: 2949432 bytes HTML transferred: 2862705 bytes Requests per second:39.24 Transfer rate: 385.75 kb/s received Connnection Times (ms) min avg max Connect:14483 Processing: 320 3291 4800 Total:321 3335 4883 -- http://www.kedai.com.my/kk http://www.kedai.com.my/eZine Strange how laughter looks like crying with no sound, and raindrops taste like tears .. without the pain ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice: using ZCatalog
If I were you, I'd wait for 2.3 final. There are some significant changes that need to make it into 2.3 yet as far as ZCatalog goes, including some textindex bugfixes and merging of textindexes on update. 2.2.5 has some Catalog bugfixes, but more bugs have been discovered and squashed since then. - Original Message - From: "Bak@kedai" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, January 15, 2001 9:06 PM Subject: [Zope] request for advice: using ZCatalog hello all i'm about to jump into uncharted waters (well, for me at least) i have a news site, and would like to use ZCatalog to present, index, etc news. all's well and good. i have a functioning ZClass, that is catalog aware, and searchable. what i'd like to know is -the stability of ZCatalog as of 2.2.5. -what's in store in 2.3 -upgradability from 2.2.5 to 2.3 -heartstopping bugs that i may have not encountered yet -performance speed/response. with my limited tests[1], i got ok response time, compared to what i have now where i iterate through objectItems/Values. but when i cache with CachePool[2], response is better. correct me if i'm wrong. i'd like to not go thru what i'm going thru now. but will this hold water? especially since i expect more requests. thanks ---8 [1] /usr/sbin/ab -n 300 -c 200 myurl This is ApacheBench, Version 1.3c $Revision: 1.38 $ apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ Server Software:Apache/1.3.9 Server Hostname:203.106.2.200 Server Port:80 Document Path: myurl Document Length:9644 bytes Concurrency Level: 200 Time taken for tests: 33.586 seconds Complete requests: 300 Failed requests:0 Total transferred: 3030864 bytes HTML transferred: 2916288 bytes Requests per second:8.93 Transfer rate: 90.24 kb/s received Connnection Times (ms) min avg max Connect:145 162 Processing: 736 14748 25346 Total:737 14793 25508 [2] /usr/sbin/ab -n 300 -c 200 myotherurl This is ApacheBench, Version 1.3c $Revision: 1.38 $ apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ Server Software:Apache/1.3.9 Server Hostname:203.106.2.200 Server Port:80 Document Path: myotherurl Document Length:9524 bytes Concurrency Level: 200 Time taken for tests: 7.646 seconds Complete requests: 300 Failed requests:0 Total transferred: 2949432 bytes HTML transferred: 2862705 bytes Requests per second:39.24 Transfer rate: 385.75 kb/s received Connnection Times (ms) min avg max Connect:14483 Processing: 320 3291 4800 Total:321 3335 4883 -- http://www.kedai.com.my/kk http://www.kedai.com.my/eZine Strange how laughter looks like crying with no sound, and raindrops taste like tears .. without the pain ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice: using ZCatalog
On Tuesday 16 January 2001 11:05, Chris McDonough wrote: If I were you, I'd wait for 2.3 final. There are some significant changes that need to make it into 2.3 yet as far as ZCatalog goes, including some textindex bugfixes and merging of textindexes on update. 2.2.5 has some Catalog bugfixes, but more bugs have been discovered and squashed since then. ok this puts me in a tight spot. any 2.3 ETA? i don't know how long i can hold this. will 2.3 be backward compatible to 2.2.x? if that's the case, it should take care of the upgrade path. and i can try and live with 2.2.5 and its bugs until 2.3 arrives. hoping that upgrading 2.2.x to 2.3.x to be as easy as upgrading from 2.1.x to 2.2.x. thanks -- http://www.kedai.com.my/kk http://www.kedai.com.my/eZine Strange how laughter looks like crying with no sound, and raindrops taste like tears .. without the pain ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice: using ZCatalog
2.3 final should be out by the end of the month. The details are at http://dev.zope.org/Resources/zope_230_plan.html Upgrading from 2.2.X to 2.3 should require minimal effort. - Original Message - From: "Bak@kedai" [EMAIL PROTECTED] To: "Chris McDonough" [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Monday, January 15, 2001 10:09 PM Subject: Re: [Zope] request for advice: using ZCatalog On Tuesday 16 January 2001 11:05, Chris McDonough wrote: If I were you, I'd wait for 2.3 final. There are some significant changes that need to make it into 2.3 yet as far as ZCatalog goes, including some textindex bugfixes and merging of textindexes on update. 2.2.5 has some Catalog bugfixes, but more bugs have been discovered and squashed since then. ok this puts me in a tight spot. any 2.3 ETA? i don't know how long i can hold this. will 2.3 be backward compatible to 2.2.x? if that's the case, it should take care of the upgrade path. and i can try and live with 2.2.5 and its bugs until 2.3 arrives. hoping that upgrading 2.2.x to 2.3.x to be as easy as upgrading from 2.1.x to 2.2.x. thanks -- http://www.kedai.com.my/kk http://www.kedai.com.my/eZine Strange how laughter looks like crying with no sound, and raindrops taste like tears .. without the pain ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice: using ZCatalog
On Tuesday 16 January 2001 12:31, Chris McDonough wrote: 2.3 final should be out by the end of the month. The details are at http://dev.zope.org/Resources/zope_230_plan.html Upgrading from 2.2.X to 2.3 should require minimal effort. ya know, this really shows why digicool really fits you guys. you guys really are cool. thanks -- http://www.kedai.com.my/kk http://www.kedai.com.my/eZine Get the tables! ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice
I started with popen2 and friends.. but that didn't seem to play well with Zope... probably mixing processes and threads is a bad idea... I dunno. Anyway I'm not sure of the best way to do this in Windows. Of course, ulitmately Shane Hathaway's idea of having a separate 'mini cvs servelet' process running that zope communicates with via tcp/ip could be the best way to add security and platform independence, but it sounds like a fair amount of extra complexity to me, esp when I don't have a lot of 'free time' to implement such a strategy. Any thoughts about this? Just the redirect idea: WinCVS (www.wincvs.org) uses some method of embedding CVS inside Windows application (IIRC they even separate CVS access methods into some DLL). The application is open source, maybe there is some idea to borrow? Warning: to say the truth, I have never read WinCVS sources. But I used WinCVS ;-) -- www.mk.maxi.pl | People resist change, regardless of what it is. Marcin.Kasperski | (Michael Reed) @softax.com.pl | @bigfoot.com | ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice
Hi Folks,
ZCVSMixin is coming along... there are several things that need to be
done, but I've been on vacation.. I'm thinking of some kind of
fishbowl proposal, but there might be something that could be done
easily to allow it to work 'as is' on Windows. (I run windows only
when I am forced to ;- ) Here is the only code in ZCVSMixin that
currently makes it fail on Windows:
#
# zope doesn't like forking... so let's just be dumb and use files
# rather than pipes..
#
stdoutFname = tempfile.mktemp()
stderrFname = tempfile.mktemp()
cmd = 'sh -v -c "%s" %s 2%s' % (command, stdoutFname, stderrFname)
zLOG.LOG('ZCVSMixin', zLOG.BLATHER, 'processing streams.. "%s"' % cmd)
iresult = os.system(cmd)
zLOG.LOG('ZCVSMixin', zLOG.BLATHER, 'checking result .. "%i"' % iresult)
resultStd = open(stdoutFname).read()
resultErr = open(stderrFname).read()
os.remove(stdoutFname)
os.remove(stderrFname)
zLOG.LOG('ZCVSMixin', zLOG.BLATHER, 'Got result.. "%s"' % resultStd)
zLOG.LOG('ZCVSMixin', zLOG.BLATHER, 'Got err result.. "%s"' % resultErr)
return iresult, resultStd, resultErr
I started with popen2 and friends.. but that didn't seem to play well
with Zope... probably mixing processes and threads is a bad idea... I
dunno. Anyway I'm not sure of the best way to do this in Windows. Of
course, ulitmately Shane Hathaway's idea of having a separate 'mini
cvs servelet' process running that zope communicates with via tcp/ip
could be the best way to add security and platform independence, but
it sounds like a fair amount of extra complexity to me, esp when I
don't have a lot of 'free time' to implement such a strategy.
Any thoughts about this?
Also.. I'm trying to find a sane way to handle folder hierarchies with
ZCVSMixin. The main problem I have is separating the attributes of
containers with their contents. Right now the export/import code
treats folderish objects as a 'clump'. I think need a sort of
'export contents' and 'export attributes' protocol so that folders
under CVS control can export their attributes (as if it were some
'content') into some 'special' file, then delegate the exportation of
their contents to the CVS Folder.
Any thoughts about this appreciated! ;-)
-steve
"Chris" == Chris McDonough [EMAIL PROTECTED] writes:
Chris Steve Spicklmire has also written the ZCVSMixin product
Chris (which I've never actually used) that might let you
Chris integration filesystem versioning with Zope more easily as
Chris well...
-Original Message- From: Patrick Lewis
[mailto:[EMAIL PROTECTED]] Sent: Friday, July 21, 2000 12:50 PM
To: '[EMAIL PROTECTED]' Subject: Re: [Zope] request for advice
On Fri, Jul 21, 2000 at 04:29:01PM +0100, Seb Bacon wrote:
Hello folks,
I'm just beginning to design my first big Zope application.
I'm writing to ask zopististitiatatas for some advice, since
I'm a bit of a newbie and I want to make sure:
[a] I reuse as much code as possible [b] I make my code as
reusable as possible [c] my application is stable and
scalable
.
___
Zope maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice
Patrick Lewis wrote: - I think it would be kind of interesting to use RCS for something like this on the backend. Build your methods in Zope to manipulate RCS, and then you get all the functionality of RCS "for free", and don't have to reinvent the wheel. You may run into problems with people not working with the most current document (I grab something from the database, Jill commits some changes to the same document, I make some changes, commit them, and destroy Jill's changes), but you would have that problem anyway as you initially described it. Checkout the ZCVS Mixin class producty thing... cheers, Chris ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] request for advice
Hello folks, I'm just beginning to design my first big Zope application. I'm writing to ask zopististitiatatas for some advice, since I'm a bit of a newbie and I want to make sure: [a] I reuse as much code as possible [b] I make my code as reusable as possible [c] my application is stable and scalable 1. Background to the application I'm building an extranet product in Zope. Much of it will be centered around a 'VersioningObject'. An example of such an object might be an image, a Word document. The objects will have a preview view associated with them. For word docs, this will be a html-ised version, for photos it will be a thumbnail. Perhaps in the future each object will also have rudimentary online editing facilities. The objects will all support versioning. This means each object can be superceded by another object of the same type. The version number will be incremented. Versions will also have Confera forums associated with them, and ACLs. Now as I see it, I want a VersioningObject to be a Folderish object that can contain a set of Versions, which are just a load of File objects of the same type. The folder has a method which slects its most recent child file object and displays it by calling its preview method. Older versions are read-only and are listed by title only. 2) My questions: i) I'm worried about the sheer volume of binary data that would accrue in the VersioningObject: a single VersioningObject might be a Word doc of 25Mb. If it has 10 versions and there are 10 such objects, that's 2.5Gb in the ZOBD. I presume this would be a bad idea performance-wise; any suggestions? ii) I'm interested in using ZPatterns because I reckon they sound like they might enhance the reusability etc. of the code; I know there have been some pointers recently, but which ZPatterns could I benefit from? or are they too arcance to worry about for now? iii) Is there any other miscellaneous advice? e.g. has anything like this been done already? Thanking you all, ladies and gentlemen. Seb. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] request for advice
On Fri, Jul 21, 2000 at 04:29:01PM +0100, Seb Bacon wrote: Hello folks, I'm just beginning to design my first big Zope application. I'm writing to ask zopististitiatatas for some advice, since I'm a bit of a newbie and I want to make sure: [a] I reuse as much code as possible [b] I make my code as reusable as possible [c] my application is stable and scalable 1. Background to the application I'm building an extranet product in Zope. Much of it will be centered around a 'VersioningObject'. An example of such an object might be an image, a Word document. The objects will have a preview view associated with them. For word docs, this will be a html-ised version, for photos it will be a thumbnail. Perhaps in the future each object will also have rudimentary online editing facilities. The objects will all support versioning. This means each object can be superceded by another object of the same type. The version number will be incremented. Versions will also have Confera forums associated with them, and ACLs. Now as I see it, I want a VersioningObject to be a Folderish object that can contain a set of Versions, which are just a load of File objects of the same type. The folder has a method which slects its most recent child file object and displays it by calling its preview method. Older versions are read-only and are listed by title only. 2) My questions: i) I'm worried about the sheer volume of binary data that would accrue in the VersioningObject: a single VersioningObject might be a Word doc of 25Mb. If it has 10 versions and there are 10 such objects, that's 2.5Gb in the ZOBD. I presume this would be a bad idea performance-wise; any suggestions? ii) I'm interested in using ZPatterns because I reckon they sound like they might enhance the reusability etc. of the code; I know there have been some pointers recently, but which ZPatterns could I benefit from? or are they too arcance to worry about for now? iii) Is there any other miscellaneous advice? e.g. has anything like this been done already? Thanking you all, ladies and gentlemen. Seb. A couple of ideas: - Don't save your data (documents, images) in the ZODB. Instead, save it to disk using the Local File System product. This will prevent the ZODB from getting too big for most OSes. - I think it would be kind of interesting to use RCS for something like this on the backend. Build your methods in Zope to manipulate RCS, and then you get all the functionality of RCS "for free", and don't have to reinvent the wheel. You may run into problems with people not working with the most current document (I grab something from the database, Jill commits some changes to the same document, I make some changes, commit them, and destroy Jill's changes), but you would have that problem anyway as you initially described it. Hope this helps. -- Patrick Lewis [EMAIL PROTECTED] ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] request for advice
Steve Spicklmire has also written the ZCVSMixin product (which I've never actually used) that might let you integration filesystem versioning with Zope more easily as well... -Original Message- From: Patrick Lewis [mailto:[EMAIL PROTECTED]] Sent: Friday, July 21, 2000 12:50 PM To: '[EMAIL PROTECTED]' Subject: Re: [Zope] request for advice On Fri, Jul 21, 2000 at 04:29:01PM +0100, Seb Bacon wrote: Hello folks, I'm just beginning to design my first big Zope application. I'm writing to ask zopististitiatatas for some advice, since I'm a bit of a newbie and I want to make sure: [a] I reuse as much code as possible [b] I make my code as reusable as possible [c] my application is stable and scalable 1. Background to the application I'm building an extranet product in Zope. Much of it will be centered around a 'VersioningObject'. An example of such an object might be an image, a Word document. The objects will have a preview view associated with them. For word docs, this will be a html-ised version, for photos it will be a thumbnail. Perhaps in the future each object will also have rudimentary online editing facilities. The objects will all support versioning. This means each object can be superceded by another object of the same type. The version number will be incremented. Versions will also have Confera forums associated with them, and ACLs. Now as I see it, I want a VersioningObject to be a Folderish object that can contain a set of Versions, which are just a load of File objects of the same type. The folder has a method which slects its most recent child file object and displays it by calling its preview method. Older versions are read-only and are listed by title only. 2) My questions: i) I'm worried about the sheer volume of binary data that would accrue in the VersioningObject: a single VersioningObject might be a Word doc of 25Mb. If it has 10 versions and there are 10 such objects, that's 2.5Gb in the ZOBD. I presume this would be a bad idea performance-wise; any suggestions? ii) I'm interested in using ZPatterns because I reckon they sound like they might enhance the reusability etc. of the code; I know there have been some pointers recently, but which ZPatterns could I benefit from? or are they too arcance to worry about for now? iii) Is there any other miscellaneous advice? e.g. has anything like this been done already? Thanking you all, ladies and gentlemen. Seb. A couple of ideas: - Don't save your data (documents, images) in the ZODB. Instead, save it to disk using the Local File System product. This will prevent the ZODB from getting too big for most OSes. - I think it would be kind of interesting to use RCS for something like this on the backend. Build your methods in Zope to manipulate RCS, and then you get all the functionality of RCS "for free", and don't have to reinvent the wheel. You may run into problems with people not working with the most current document (I grab something from the database, Jill commits some changes to the same document, I make some changes, commit them, and destroy Jill's changes), but you would have that problem anyway as you initially described it. Hope this helps. -- Patrick Lewis [EMAIL PROTECTED] ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
