[Zope-dev] Session
How can I use this line ? Do I need any product ? Anderson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Session
Hi Anderson, Anderson Ami wrote: > How can I use this line ? Do I need any product ? > > I don't understand your question. Perhaps you need to explain your situation in more detail. In any case, this looks like a question about using Zope rather than about the development of zope itself, so I think you should ask only on the [EMAIL PROTECTED] list, rather than posting to both the [EMAIL PROTECTED] and [EMAIL PROTECTED] lists. There are very few reasons that anyone should post the same message to both [EMAIL PROTECTED] and [EMAIL PROTECTED] There's even a message that the mailing list software puts at the end of each message: ** No cross posts or HTML encoding! ** ^^ Please take note for the future. Also, you seem to be missing a " character at the end of the dtml you posted. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
Michael Bernstein wrote: > Jimmie Houchin wrote: > > > > I would like to echo Michael's sentiment and comments. I am at the > > beginning stage of Zope development of a website which will have > > millions of objects of a single object type and multiple of such. > > are your objects intended to be indexed by ZCatalog as well, > or are you planning some other method of finding your > objects? Yes as far as I know at this time. I am developing off of a cvs build. I keep it updated regularly. This provides me with the latest in ZCatalog and other development. > > [snip stuff about mounted databases] > > > > I think there are many who would like to keep their development within > > Zope using objects. Any information on best use or development > > strategies for such setups would be greatly appreciated. > > Thanks for adding your voice! Your welcome. :) > I also think that applications such as we're discussing > should be possible to deploy using nothing but Zope's built > in capabilities. I understand that Zope has certain > limitations in situations that require many writes to the > database, and I accept those limiattions as being a > neccessary trade-off for a storage strategy that uses an > appending file format. I am still trying to determine if > Zope (especially using the two development options I > outlined) has any built in limitations regarding the number > (or number x size) of objects stored. I agree and think yours is a good and valid question. > I really appreciate Zope's features (especially with > ZPatterns) that allow applications to be developed that are > storage-agnostic, and I feel that this is especially useful > for tying into existing legacy systems, but I don't want to > develop a new application, storing new data, that is tied to > a specific storage methodology such as an RDBMS. Agreed. It also should be recognized that RDBMSes are not without issues and problems of their own, especially when you are dealing with objects. > People who wish to customize an application to leverage > their existing legacy data should be free to do so, but I've > noticed that Zope products that have some external system as > a pre-requisite (Worldpilot, ZCommerce, etc.) are deployed > far less often than those which do not (Squishdot, Zwiki, > etc.). > > So, again: Has anyone run up against any performance or > other limitations regarding large numbers (hundreds of > thousands or more) of objects stored within the ZODB either > in a BTree Folder or a Rack? > > In other words, will the system slow down if you add enough > objects? If the general consensus came back and said that currently the ZODB is inappropriate for such magnitude, the next place I would look is into Forethought's OODBMS implementation 4ODS. It would require additional work such as installing whatever backend you wish to use with it. It currently works with I believe Oracle and PostgreSQL. Other backends are or will be developed. But development is OO. I don't know if anyone has used or considered using 4ODS with Zope. Could provide interesting options. But I'm for all Zope first. Jimmie Houchin > Thanks, > > Michael Bernstein. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
I don't know if this was mentioned earlier, but you might want to check out the OracleStorage product which allows you to use Oracle as storage for the ZODB. http://www.zope.org/Products/OracleStorage This solution would likely scale to your needs. Also there are several object databases like Matisse and Objectivity that might be worth considering. Good luck! -- | Casey Duncan | Kaivo, Inc. | [EMAIL PROTECTED] `--> ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Specialists and __bobo_traverse__
I'm still lost as to when __bobo_traverse__ is checked for object traversal and when not. I have very simple piece of SkinScript: WITH SELF COMPUTE __bobo_traverse__ = traversal_method. Looking at the traverse method in BaseRequest.py it looks to me as if an object is always checked if it has a __bobo_traverse__ method but this is not what is happening. If I manually type in a url, say path.to.specialist/dataskin1, where dataskin1 exists, my traversal_method is not called. When I type in path.to.specialist/dataskin2, where dataskin2 does not exist then my traversal_method is called. Noticing that the traversal_method is called when an object can not be found I tried path.to.specialist/dataskin1/dataskin2, where dataskin1 is in the specialist's rack but dataskin2 is in another specialist's rack and will therefor not be found. In this case my traversal_method was not called. If it was then I could retrieve it from the appropriate specialist. I hope the above is clear because I'm somewhat desperate at the moment. If I'm trying the impossible, a simple "forget about it" will also do. Roché ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Specialists and __bobo_traverse__
It sounds to me like you're trying to have SkinScript supply a bobo_traverse for the Specialist itself, and this you cannot do. (There are other ways to change a Specialist's bobo_traverse behavior, but SkinScript is not one of them.) The SkinScript you're doing is meaningful only if you want to go to specialist/skin1/something, in which case your traversal_method should be called to process "something". I don't have the slightest idea what you're trying to accomplish here - I probably missed the beginning of this thread. Do you want to change the way the specialist processes the string "dataskin1", or the way dataskin1 processes the URL component that comes next? Each requires a different approach. At 07:28 PM 1/16/01 +0200, Roch'e Compaan wrote: >I'm still lost as to when __bobo_traverse__ is checked for object traversal >and when not. > >I have very simple piece of SkinScript: WITH SELF COMPUTE __bobo_traverse__ >= traversal_method. Looking at the traverse method in BaseRequest.py it >looks to me as if an object is always checked if it has a __bobo_traverse__ >method but this is not what is happening. > >If I manually type in a url, say path.to.specialist/dataskin1, where >dataskin1 exists, my traversal_method is not called. When I type in >path.to.specialist/dataskin2, where dataskin2 does not exist then my >traversal_method is called. Noticing that the traversal_method is called >when an object can not be found I tried >path.to.specialist/dataskin1/dataskin2, where dataskin1 is in the >specialist's rack but dataskin2 is in another specialist's rack and will >therefor not be found. In this case my traversal_method was not called. If >it was then I could retrieve it from the appropriate specialist. > >I hope the above is clear because I'm somewhat desperate at the moment. If >I'm trying the impossible, a simple "forget about it" will also do. > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] manage_changeProperties in a loop
Suppose I have a list of existing folder objects and depending on their index in the list I want to set a property called 'rang'. I assumed you could something like this: (normally the list comes from outside) Any ideas or hints? Thanks Arno Gross, email: [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
> While that would work for the simple object case, I find the > prospect of storing a bunch of BLOBs (for the image data of > the Photos) in an RDBMS to be *most* un-appetizing. Storing > them on the server's file-system seems in-elegant as well. Okey dokey, just a suggestion. I have heard people talk about large ZOBD's but once I go over a 10,000 object mark I just find a RDBMS easier myself. Go for it good luck! > Does anyone know of any hidden 'gotchas' when dealing with > this many objects, regardless of the hit-load on the system? Mostly starting and stopping Zope, the 2gb limit (which can be avoided), pulling objects back out with complicated queries are my biggest gripes. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] manage_changeProperties in a loop
Arno Gross wrote: > Suppose I have a list of existing folder objects and depending > on their index in the list I want to set a property called 'rang'. > > I assumed you could something like this: > > > > (normally the list comes from outside) > > > Here, you're getting an item with the literal id 'sequence-item'. What you mean is to get the item with the id described by the variable sequence-item. So, use: Hope that helps. This really is a [EMAIL PROTECTED] question, not a [EMAIL PROTECTED] question. Please do try to choose the appropriate list for the question. Thanks. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
On Mon, 15 Jan 2001, Michael Bernstein wrote: > as Squishdot). Adding a dependency on an RDBMS or requiring > additional setup on the server's FS seems a step in the > wrong direction. [...] > So the question remains: Will either approach (within the > ZODB) allow me to scale the application to hundreds of > thousands (or even millions) of objects indexed in a > ZCatalog? [...] > I know that the ZCatalog/ObjectManager approach used by > Squishdot will scale to over 9,000 objects (the number of > postings to date at technocrat.net), So I'm reasonably > certain that my proposed ZCatalog/BTree Folder approach will > be at least as scalable. I'm slightly less confident about > the Specialist/Rack approach, because I don't know of any > sites that have used them to store that many objects in the > ZODB, but only slightly. My understanding is that the point of ZPatterns is to hide the data storage implementation from the application. You refer to this in another post. The point being that you can *change your mind* later, with minimal disruption to your application. Not only that, but people who *use* your product can make their own decision about where to store the data. So by using ZPatterns you would have a product that could be installed without any addons other than ZPatterns itself, and yet let the users of your product use an RDBMs if that works better for them. In addition, it seems to me that your comments about ZCatalog+BTree apply equally well to ZPatterns, since you can use the Catalog to index stuff stored in a rack through the use of appropriate triggers, and it is my understanding that the default in-ZODB rack storage uses BTree internally. So overall I'd think going with ZPatterns would be the right decision. It gives you and others using your code the maximum flexability. Unfortunately I don't have much input on your question about real-life scalability...the most I've done is stored 6 small objects in a hierarchy of zope folders, indexed by the catalog, with no perceptable slowdown in search or retrieval speed. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Specialists and __bobo_traverse__
> It sounds to me like you're trying to have SkinScript supply a > bobo_traverse for the Specialist itself, and this you cannot do. (There > are other ways to change a Specialist's bobo_traverse behavior, but > SkinScript is not one of them.) The SkinScript you're doing is meaningful > only if you want to go to specialist/skin1/something, in which case your > traversal_method should be called to process "something". This is exactly what I want to do ie. process "something". > I don't have the slightest idea what you're trying to accomplish here - I > probably missed the beginning of this thread. Do you want to change the > way the specialist processes the string "dataskin1", or the way dataskin1 > processes the URL component that comes next? Each requires a different > approach. The best way to describe this is with an example I suppose. I have a specialist "Products" and nested within "Products" is another specialist "Modules". The current status quo: I go to a Product dataskin with a url like Products/. I go to a Module dataskin with a url like Products/Modules/ Where I want to be: The url to the Product dataskin stays the same. I go to a Module dataskin with url like Products// As I see it the is like the "something" you mentioned above. The module dataskin would obviously not be found by the Products specialist, and in that case I simply want to tell it to go look in the Modules specialist for the module. I hope this makes more sense, if not I'll be glad to explain more. Many thanks Roché ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] ZCatalog with ZClass
Hello, everyone, It seems that many of you have used ZCatalog pretty well. My problem is that I'd like a ZClass (of a Product) to subclass CatalogAware, and implement all the index and search inside the Product. That means I don't want to do any DTML method stuff outside of the Product. Could anyone tell me how to do it? I can't find anything like API document. I don't have to read through the "Catalog.py" stuff, do I? Thanks in advance. Cheers... Lily ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
Michael Bernstein wrote: > So, again: Has anyone run up against any performance or > other limitations regarding large numbers (hundreds of > thousands or more) of objects stored within the ZODB either > in a BTree Folder or a Rack? I was looking into the same issues recently, but for a much smaller set of data (5ish). In my tests ZPatterns/binary-trees scaled well for storage and retrieval. But ZCatalog did not. It was basically useless for partial matching searches (taking many minutes for searches that retrieved more than 100 matches). I was also concerned about the indexing overhead. It doesn't scale well when changing/adding many things at a time (we might have bulk adds/changes). I ended up deciding to go with a RDBMS backend for data storage with a ZPatterns interface. SkinScripts work so well for this that I'm actually glad I switched. It simplified my design and implementation immensely. -- John Eikenberry [[EMAIL PROTECTED] - http://zhar.net] __ "A society that will trade a little liberty for a little order will deserve neither and lose both." --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
Does ZPatterns provide a nice interface / way for storing classes in a RDBMS? I have to say using an RDBMS is not as transparent as I would like, this may may improve it. Finally a reason for me to ZPatterns... -- Andy McKay. - Original Message - From: "John Eikenberry" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Cc: "Michael Bernstein" <[EMAIL PROTECTED]> Sent: Tuesday, January 16, 2001 3:22 PM Subject: Re: [Zope-dev] Massive scalability > Michael Bernstein wrote: > > > So, again: Has anyone run up against any performance or > > other limitations regarding large numbers (hundreds of > > thousands or more) of objects stored within the ZODB either > > in a BTree Folder or a Rack? > > I was looking into the same issues recently, but for a much smaller set of > data (5ish). In my tests ZPatterns/binary-trees scaled well for storage > and retrieval. But ZCatalog did not. It was basically useless for partial > matching searches (taking many minutes for searches that retrieved more > than 100 matches). I was also concerned about the indexing overhead. It > doesn't scale well when changing/adding many things at a time (we might > have bulk adds/changes). > > I ended up deciding to go with a RDBMS backend for data storage with a > ZPatterns interface. SkinScripts work so well for this that I'm actually > glad I switched. It simplified my design and implementation immensely. > > -- > > John Eikenberry > [[EMAIL PROTECTED] - http://zhar.net] > __ > "A society that will trade a little liberty for a little order > will deserve neither and lose both." > --B. Franklin > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] 2.3 beta
On a more minor note would it be possible to get rid of the old style dtml syntax (
Re: [Zope-dev] Massive scalability
John Eikenberry wrote: > > Michael Bernstein wrote: > > > So, again: Has anyone run up against any performance or > > other limitations regarding large numbers (hundreds of > > thousands or more) of objects stored within the ZODB either > > in a BTree Folder or a Rack? > > I was looking into the same issues recently, but for a much smaller set of > data (5ish). In my tests ZPatterns/binary-trees scaled well for storage > and retrieval. But ZCatalog did not. It was basically useless for partial > matching searches (taking many minutes for searches that retrieved more > than 100 matches) Was this true even for cases where the batch size was smaller than 100? For example, if a search returns over 100 results but the batch size is only 20 (so that only 20 results at a time are displayed), do you still get the performance hit? > [snip] > I ended up deciding to go with a RDBMS backend for data storage with a > ZPatterns interface. SkinScripts work so well for this that I'm actually > glad I switched. It simplified my design and implementation immensely. So you're saying that you are doing all searching using SQL statements, and not just object retreival and storage, correct? How are you handling full text searches? Cheers, Michael Bernstein. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
Andy McKay wrote: > > > While that would work for the simple object case, I find the > > prospect of storing a bunch of BLOBs (for the image data of > > the Photos) in an RDBMS to be *most* un-appetizing. Storing > > them on the server's file-system seems in-elegant as well. > > Okey dokey, just a suggestion. I have heard people talk about large ZOBD's > but once I go over a 10,000 object mark I just find a RDBMS easier myself. > Go for it good luck! Thanks! I appreciate different points of view on this problem, even if you have different 'comfort zones'. > > Does anyone know of any hidden 'gotchas' when dealing with > > this many objects, regardless of the hit-load on the system? > > Mostly starting and stopping Zope, [snip] Are you saying that Zope's startup and shutdown time is affected by the size of the ZODB? Thanks, Michael Bernstein. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
RDM wrote: > > On Mon, 15 Jan 2001, Michael Bernstein wrote: > > as Squishdot). Adding a dependency on an RDBMS or requiring > > additional setup on the server's FS seems a step in the > > wrong direction. > [...] > > So the question remains: Will either approach (within the > > ZODB) allow me to scale the application to hundreds of > > thousands (or even millions) of objects indexed in a > > ZCatalog? > [...] > > I know that the ZCatalog/ObjectManager approach used by > > Squishdot will scale to over 9,000 objects (the number of > > postings to date at technocrat.net), So I'm reasonably > > certain that my proposed ZCatalog/BTree Folder approach will > > be at least as scalable. I'm slightly less confident about > > the Specialist/Rack approach, because I don't know of any > > sites that have used them to store that many objects in the > > ZODB, but only slightly. > > My understanding is that the point of ZPatterns is to hide > the data storage implementation from the application.[snip] > The point being that you can *change your > mind* later, with minimal disruption to your application. Not > only that, but people who *use* your product can make their > own decision about where to store the data. So by using ZPatterns > you [...] let the users > of your product use an RDBMs if that works better for them. Very good points, and ones that I will keep in mind. Thanks. > In addition, it seems to me that your comments about ZCatalog+BTree > apply equally well to ZPatterns, since you can use the Catalog > to index stuff stored in a rack through the use of appropriate > triggers, and it is my understanding that the default in-ZODB > rack storage uses BTree internally. I do not know if BTree folders and Racks share the same B-Tree implementation, which is why I qualified my statement as 'slightly less confident'. > Unfortunately I don't have much input on your question about > real-life scalability...the most I've done is stored 6 small objects > in a hierarchy of zope folders, indexed by the catalog, with > no perceptable slowdown in search or retrieval speed. Hmm. John Eikenberry mentioned a slowdown with about 50,000 objects on partial-match searches, but I don't know how simple/complex the objects were, or how many atributes were being indexed. How many indexes of various types was your ZCatalog maintaining on your objects? Thanks, Michael Bernstein. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
Andy McKay wrote: > Does ZPatterns provide a nice interface / way for storing classes in a > RDBMS? I have to say using an RDBMS is not as transparent as I would like, > this may may improve it. Finally a reason for me to ZPatterns... The best way to get a taste is to try it out. The easiest way to do this is to install LoginManager plus the DB/DA. Then follow the instructions in... LoginManager with SQL and Skinscript http://www.zope.org/Members/dlpierson/sqlLogin I had it up and running pretty quickly following these instructions. More generally the combo of sql-methods and ZPatterns (ie. skinscripts) seems pretty compelling. I've used SQLMethods pretty extensively for the past 18 months. They have definate limitations which is why I'm working on the object based system. The ZPatterns abstraction seems to provide the best of both worlds. You get the nice parts of SQLMethods; timed cache, dtml query syntax and web viewable sql querries, plus you get a nice object abstractaction (much better than plugable brains). SkinScripts allow for easy attribute and trigger handling, basically like a simple object description language. If you can't tell, I'm pretty sold on ZPatterns. And once deciding that an RDBMS was the best way to go for data storage, it fit into the 'pattern' very nicely. I haven't deployed it yet, but its pretty fun to work on. :) -- John Eikenberry [[EMAIL PROTECTED] - http://zhar.net] __ "A society that will trade a little liberty for a little order will deserve neither and lose both." --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog with ZClass
Lily Li wrote: > > Hello, everyone, > > It seems that many of you have used ZCatalog pretty well. > > My problem is that I'd like a ZClass (of a Product) to subclass > CatalogAware, and implement all the index and search inside the Product. > That means I don't want to do any DTML method stuff outside of the Product. What you probably want is to create a repository that subclasses ZCatalog as is described in this HowTo: http://www.zope.org/Members/tseaver/inherit_ZCatalog And then add your CatalogAware ZClasses into it. Then you can set up your ZCatalog/ObjectManager with search methods. HTH, Michael Bernstein. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
Michael Bernstein wrote: > John Eikenberry wrote: > > > > I was looking into the same issues recently, but for a much smaller set of > > data (5ish). In my tests ZPatterns/binary-trees scaled well for storage > > and retrieval. But ZCatalog did not. It was basically useless for partial > > matching searches (taking many minutes for searches that retrieved more > > than 100 matches) > > Was this true even for cases where the batch size was > smaller than 100? For example, if a search returns over 100 > results but the batch size is only 20 (so that only 20 > results at a time are displayed), do you still get the > performance hit? Short answer: yes Long answer: If you check out the source and/or hit it with the profiler you'll see that the way the partial search works is to first do a more general search then to limit the hits as much as possible via regex's. Both these steps have to happen no matter the batch size, and this is where you take the performance hit. > > [snip] > > I ended up deciding to go with a RDBMS backend for data storage with a > > ZPatterns interface. SkinScripts work so well for this that I'm actually > > glad I switched. It simplified my design and implementation immensely. > > So you're saying that you are doing all searching using SQL > statements, and not just object retreival and storage, > correct? How are you handling full text searches? Yes. I'll use MySQL's built in pattern matching facilities. It can do full text searches with partial matching, and it can do this fast. I'm working on a system that will return the DataSkin's in responce to the query. Allowing me to deal with just the objects yet use all of MySQL's facilities. I'v just started to work on this as part of a larger project, but I'm doing it full time and should have something fairly soon. My company is very free software friendly, so I'll be able to share it once its ready. If you happen to be interested. -- John Eikenberry [[EMAIL PROTECTED] - http://zhar.net] __ "A society that will trade a little liberty for a little order will deserve neither and lose both." --B. Franklin ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
On Tue, 16 Jan 2001, Michael Bernstein wrote: > I do not know if BTree folders and Racks share the same > B-Tree implementation, which is why I qualified my statement > as 'slightly less confident'. I'm fairly certain that BTree folders, Squishdot, and Racks all use the BTree module that comes with zope, each in their own way. (Note that that module has some issues with DB activity on writes, which I believe someone is eventually going to fix...) > Hmm. John Eikenberry mentioned a slowdown with about 50,000 > objects on partial-match searches, but I don't know how I have not experiemented with partial match searches, although I did build the site with that type of index. > simple/complex the objects were, or how many atributes were > being indexed. How many indexes of various types was your > ZCatalog maintaining on your objects? The catalog indexed just a few pretty short text properties, and searching is really done only one one (the longest) field. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 2.3 beta
... and what's the chance of getting my minor (but incredibly nice) change to the management menus in? I'd rather not have to ship a modified version of Zope... Richard ps. for those who missed the first post, I'm advocating that the ordering of the management menu tree be done by the ObjectManagers and their ilk rather than an enforced alphabetic sort in the tree itself. More logical groups exist in some cases. -- Richard Jones [EMAIL PROTECTED] Senior Software Developer, Bizar Software (www.bizarsoftware.com.au) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Massive scalability
John Eikenberry wrote: > > > Long answer: If you check out the source and/or hit it with the profiler > you'll see that the way the partial search works is to first do a more > general search then to limit the hits as much as possible via regex's. > Both these steps have to happen no matter the batch size, and this is where > you take the performance hit. Can you tell us a bit about how many indexes (and what types) you were maintaining about each object? Another poster reported no problems with 60,000 objects. > > So you're saying that you are doing all searching using SQL > > statements, and not just object retreival and storage, > > correct? How are you handling full text searches? > > Yes. I'll use MySQL's built in pattern matching facilities. It can do full > text searches with partial matching, and it can do this fast. I'm working > on a system that will return the DataSkin's in responce to the query. > Allowing me to deal with just the objects yet use all of MySQL's > facilities. Sounds interesting. > I've just started to work on this as part of a larger project, but I'm doing > it full time and should have something fairly soon. My company is very free > software friendly, so I'll be able to share it once its ready. If you > happen to be interested. Sure, keep me posted. Thanks, Michael Bernstein. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] ANNOUNCE: Zope 2.3.0 beta 1 released
Hello all, Zope 2.3.0 beta 1 is now available. You can download it from Zope.org: http://www.zope.org/Products/Zope/2.3.0b1/ This release contains the final new features for 2.3, including: - A browser preferences screen to allow people to tweak certain aspects of the new management UI to their liking. For the folks who complained that they didn't like the new top frame, they can (among other things) turn it off from the browser preferences screen. - Michel and Amos' new QuickStart material. This is different in spirit than the old "QuickStart" folder. The new material is meant as a handy high-level set of pointers to important learning materials. The old "QuickStart" folder is still around by default for the beta release - we will need to decide whether to get rid of it altogether or whether it still has value as an example and should be cleaned up and retained. - The logout function has been implemented in a fairly minimal way. We will try to make this nicer by final if we get time. - Chris Petrilli has cleaned up the ZCatalog interface and done some logical reorganization to move some things onto an Advanced tab. For more information on what is new in this release, see the CHANGES.txt and HISTORY.txt files for the release: - http://www.zope.org/Products/Zope/2.3.0b1/CHANGES.txt - http://www.zope.org/Products/Zope/2.3.0b1/HISTORY.txt Note that now we are in the beta period, so binaries are now available for all of the usual platforms. Enjoy! Brian Lloyd[EMAIL PROTECTED] Software Engineer 540.371.6909 Digital Creations http://www.digicool.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
