Re: [Zope-dev] RAMCacheManager and gzip
> There is zlib support in Python, and it is used in Medusa already. ZServer > does not call it, but the compression stuff is there. > > There is one good reason not to use zlib though, Micro$oft Internet > Distorter does not support content-encoding: gzip and for content-encoding: > compress it expects data to be compressed using the Unix compress tool... (I > think someone reported this already... oh well) Hmm, how does mod_gzip compress it then? In every request I get from IE includes the "accept-encoding: gzip" string, and apache is sending it gzip-compressed back. Do you mean that the mod_gzip is compressing it different then zlib? mart(ijn) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: more on the segfault saga
Hello,
We also have the problem with zope crashing with segfault 11.
I've read al earlier posts on this subject, but couldn't notice anything
that could arrange a direct solution, or I should have missed something?
The only thing I can try is to describe our situation :
- We have a production server running zope 2.5.1 beta (binary release,
linux), so the crash cannot be from the bugs listed in Matthews
Stability Howto.
- The crash occurs as well on Windows as on Linux, (both binary releases
2.5 beta 1, as well 2.5.0 as well 2.4.3 on both platforms)
- It happens only with "alot" of traffic ("alot" is more then 40
kbytes/sec, 500 users on a intranet, I hope I don't have to scale zope
with this little amount of traffic?)
- It DOESN't crash with the -t 1 option, so the crash has something to
do with threading. However, the performance is very poor with this
option, so this is not a good solution.
- We use a flash generator c extension module (ming), but with this
removed, it still crashes, so it's not a non-standard-zope c extension
module that can cause the crash.
- We DON't use any DA for relational access, so all earlier suggestions
that the mysql/oracle DA modules are causing the crashes should be
reconsidered
- ZOPE_SECURITY_POLICY=PYTHON is not taking away the crashes.
- We use our own python Products which are accessed when the crash
occures, but these products are very clean and totally written in
Python, so that should not be any problem I suppose, because Zope
doesn't crash in single thread mode. We also use alot of Python Scripts,
but these bugs are already fixed? (says the Stability Howto)
I don't know where to start, because attaching GDB doesn't make any
sense, since you have to start zope single threaded (according to Matts
Stability Howto) and then no crashes occur.
Is this problem solved if I install python 2.2 for example? Are there
any bugfixes in this release from Python 2.1.2 ?
I don't know what the status is right now? Is zope corp. working on it
trying to find the bug? Can I be of any help tracking down this bug?
regards,
Martijn Jacobs
East Site
[EMAIL PROTECTED]
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: more on the segfault saga
> Actually, at least in Linux, with a recent gdb, you can attach gdb to > zope in multithread mode. Just take the "-t 1" from the command line > sugested by the StabilityHOWTO and you're set. Best results are achieved > by compiling everything from source (python even, use the > --prefix=/path/to/zope-src so as not to mix up with your installed > python and be careful to use this python when installing zope) and > running: Ok, I succeeded tying up the gdb on the production server. I have to wait until tomorrow for results, because in the evening the intranet is not used by the specific company :) Tomorrow it will crash for sure, because it crashes about 20/30 times a day, so then I will post the results as soon as possible! It's very frustrating that we cannot reproduce this bug in out own environment, whatever we try. (all workstations requesting like hell, but we cannot succeed crashing it!) It's very nice to hear that you people are trying to solve the problem, also thanks to the guys from Zope Corp. who are spending there time for it! Hope the bug will be resolved soon. kind regards, martijn jacobs ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: more on the segfault saga
Hello again, I have a little bit more information about our specific "crash" environment. First of all, I looked in the Big M Log at the time of the crashes, but couldn't find anything interesting : 8 of the 10 times it was a dtml document containing plain stylesheet data :( I don't think there's any usefull information in here, because the crashes are threading related. Second of all I managed to get a traceback (?) of a gdb session, I don't know if it's any usefull information, I'm not a zope expert on this area, I hope other people (zope developers?) understand what's on the traceback. The customer was not happy with my session, but that environment is the only one that crashes, we still cannot reproduce the error here, nor can we make a testcase for the crash there :( I put the traceback online on http://www.coherence.nl/crash.txt. regards, martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: more on the segfault saga
Sorry, the correct URL is http://www.coherence.nl/crash.txt (without the dot) martijn. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: more on the segfault saga
Hello Leo, Matt, Brian, > I'm on it. Will send results when they're available. If anyone wants > to talk to me during the period, I'll be on IRC. If you need any assistance for anything, I'm at your service Which channel/server are you on IRC? Did somebody succeed reproducing the crash? We try the best we can to make a reproducable testcase, but Zope doesn't want to crash here... The clients who use the production Zope which crashes are all using Active Desktop (I know :( ), could that be of any matter? Technically it's insane if it does matter, but you never know... I'm out of capabilities right now, don't know what to do anymore, so I hope the bug will be found soon. regards, martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Fwd: [Zope-dev] Re: more on the segfault saga]
> Hello segfaulters and others interested in Zope instability issues! Hello again :) > Our demi-god Matt Kromer from ZopeCorp has come up with a possible way > to corner the instability issue AND give you a stable, cycle-garbage > collecting Zope. We probably should build a statue for this guy somedays > Since the problem seems, so far, to be caused by the Python Restricted > Compiler (which is used in everything from dtml expressions to python > scripts to other stuff) not completing fully collectable objects > before the Python cycle garbage collector finds them, the solution > is to lock > out the gc while creating these objects. The only easy way to do this > currently is to disable the automatic gc and run manual garbage > collections only when we're pretty sure no one else is running, and at > the same time not letting anyone else run when we're running the gc. Just to make sure : I put gc.disable() in Z2.py and apply this ZServerPublisher.py patch? Or do I have to add some more code to let the garbage collector collect more often or less or... > We're close guys, very close. You can't imagine how glad I would be if this thing is fixed. Thanks again for all your efforts! Martijn Jacobs East Site ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Fwd: [Zope-dev] Re: more on the segfault saga]
Hello Leo! > Tell us if your segfaults go away and whether Zope performance is > impacted by it. The segfault is AWAY! I have an uptime of 2 hours now, it has never been so long since it's in use! I don't notice any performance differences, but I also have to say that I can't measure it really. The more important is that it doesn't crash anymore! Is this a *final* solution for the crashing? Or is this just a temporary solution to make sure where the problem is? BTW : Thank Matt for the patch! regards, Martijn. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
re: [Zope-dev] Crashing problem? Python workaround?!
Hi Matt, Leo > Pythonlabs found this today after much assistance from the folks on > #zope on the irc.openprojects.net system; particularly, Leonardo Rochael > Almeida's assistance has been extremely valuable in isolating this > problem. And jus to say it again : thank you all again!! You've made a customer very happy again! > This is an intermediate workaround until Pythonlabs develops an official > fix. However, by using this workaround, you do not need the specially > modified ZServerPublisher module I coded. Will this patch be included in the next Zope binary releases? I could build a zope for on a particular environment, but it would be alot easier to upgrade all customers with the new binary release. Anyway : In other words : if this is added in the official Python release, the bug is cleanly resolved? -- martijn jacobs east site ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
re: [Zope-dev] ding, dong, the segfault is dead. Patch attached
Hello leo, > I just put this patch up in production environment and it's been up for > more than 1h30m now, which for me is enough indication that it's in fact > the final solution to the problem. Nice to hear. Is there any change that this patch will be included in the next Zope Binary release? I would love to apply this patch on a python source, compile it, and put it away, but you don't want to do that on a 28K8 PC Anywhere connection, without any compilers installed. So I'm hoping on a very soon Zope release with this version of python. The patch from Matt is OK, but the peformance of heavy database access is dramatic, so I need to upgrade real soon. The beast is killed! Finally! :) Martijn. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZMI / JavaScript brainstorm
What is wrong with editing the files trough FTP / WebDAV? I've mounted a complete zopedatabase on a directory, and you can edit all files with your favorite editor. Why wanting such editing posibilities in a easy to use , in everybrowser working interface like the ZMI? ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZClasses useful! [Was: ZMI / JavaScript brainstorm]
> I will not stop dissing them until people stop using them and then complaining > when they break, don't do what they want or behave unexpectedly. What's your efford on this task? If people like to use them, let them! It's not up to you to decide for THEM what's best or not. The choice for YOURSELF not to use them is just as fair as other peoples choices. martijn. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Slow zope on windows 2000?
Hello Casey, > One thing to remember is that running multi-treaded Python apps on a > multi-processor box is suboptimal unless you can bind all the threads to a > single processor, due to the Python GIL. IOW : Buying a dual processor computer is not so usefull if the computer is dedicated running zope? And if you have to use a dual CPU computer, binding all the threads to a single processor increases zope performance? (If so, how can you do that? :) > The only way to do a true comparison would be to run the same tests on the > same app on the same hardware with the same load but with different OSes. That's true. I was wondering if you guys at zope comp. have any recent benchmarks yourself with the different Operating systems? Martijn. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Please help the zope.org improvement efforts
2. People to step up and own pages or sections. This means contributing content! I've volunteered a while ago but haven't found the time to do what I wanted to do. I will make time for this the following two / three weeks and write some content. Martijn. -- Martijn Jacobs Four Digits, Internet Solutions ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZCatalog sorting issue
Wat denk je hiervan? Hello. In zope 2.10.5 (and probably 2.10.6 and 2.11 and, as I've read, all releases above 2.7) we've encountered a sorting bug in a dtml-in call when querying the catalog. I don't think it's dtml only related, but I'm not sure about that. It is the same bug as found on : https://bugs.launchpad.net/zope2/+bug/143504 Adding def __cmp__(self, other): return 0 to Products.ZCatalog.CatalogBrains.AbstractCatalogBrain seems to correct the problem though and I was wondering why this isn't added in the zope core. Does it break other stuff or should the problem fixed somewhere else? Somebody has some thoughts? Regards, Martijn -- Martijn Jacobs Four Digits, Internet Solutions a: Willemsplein 15-1 6811 KB Arnhem NL kvk: 09162137 | btw: 8161.22.234.B01 e-mail: [EMAIL PROTECTED] | web: http://www.fourdigits.nl tel: +31 (0)26 44 22 700 | fax: +31 (0)84 22 06 117 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] "Overriding Products"
Hello guys. I have a question regarding product importing in zope 2.7.x, as since zope 2.7 this part is way more flexible then it was before. I'd like to accomplish the next situation : - A Zope base installation has Product X installed in /lib/python/Products which I want to change or develop further - x instances of Zope use the produtc from this base install. - At one instance I want to test and develop, without breaking the other instances. - Copying the Product X folder to the /Products folder is not an option, as import conflict errors occure if you do not remove the product from the base installation. - Copying a local version in all 4 instances is a solution, but to manage that will cost alot of work, especially if you have alot of instances. - What I really want is that you can copy the Product X folder to the Products (or "CustomProducts" for example) of an zope instance, and that zope will figure out itself that it should import that version of the product instead of the copy in the base installation. 'Overriding' base products is the name you could give this situation. I tried alot myself, for example checking in the __init__.py if there is a copy of this product in /CustomProducts and then importing that product instead of my own, but I always get into trouble : AttributeError: 'module' object has no attribute '' It doesn't have to work for all products, only for some of the products that we are developing ourselfs. We also have a different structure than normal products as following : Main Product - Sub product X - Sub product Y In the __init__.py of Main product we import and initialize the subproducts. The idea is to put one of those subproducts in a zope instance/CustomProducts folder for further development and that the other instances of zope still uses the base version of the subproduct. I hope you can help, thanks in advance for your time. Kind regards, -- Martijn Jacobs East Site | Postbus 642 | 6800 AP Arnhem t: +31 (0) 26 35 14 295 | f: +31 (0) 26 44 20 432 @: [EMAIL PROTECTED] | w: www.eastsite.nl ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Folderish or SimpleItem object types for structural content
Hello guys! I have a simple question about the design of hierarchical zope product structures in Zope 2. For example, you have categories, within categories you have sub-categories, within sub-categories you have sub-sub categories, within that you have content items, within content items you haveetcetera. All those products are different 'types', with different functionality and different 'roles' as what they represent, so I write different products for every type. For every object that can contain subobjects you could derive it from 'Folder' (or Object Manager). But when you derive all these product types from 'Folder' you end up with ALOT of objects in your database. In a lot of real world cases you have categories and items, that makes things easy, but what about the structure explained above? My question for you guys is : When do you decide to make it a Folderish object which contains sub-objects, or to make it a Simple Item object with, for example, some structured XML content, or dictionary / list properties containing the 'sub-structure'? The last option is more work, because you have to write your add/edit/delete/etc code yourself, writing it with Folders makes it more easily, but then you have an explosion of objects in the ZODB. I would like to have some fresh thoughts about this, so maybe you can help. Thank you in advance, kind regards, Martijn. -- Martijn Jacobs Four Digits, internet solutions e-mail: [EMAIL PROTECTED] | web: http://www.fourdigits.nl tel: +31 (0)26 44 22 700 | fax: +31 (0)84 22 06 117 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Folderish or SimpleItem object types for structural content
Hello Dieter. ...For every object that can contain subobjects you could derive it from 'Folder' (or Object Manager). But when you derive all these product types from 'Folder' you end up with ALOT of objects in your database. It that a problem? It might become one once you get millions of objects... Usually, I would expect that most objects come from you leaves and not from the classification structure... That will be true, but I was wondering where to draw the line, for example with perfomance issues. All the objects will be in a ZCatalog, to allow searching for example. But one 'ObjectManager' step less in the choice what to do with the products results in much less objects in the catalog. The estimation about the amount of objects, with only the leave nodes as 'SimpleItem' objects will be 30.000. I never had so many objects in a ZODB, that's why I'm asking this, but as your answer suggests it's still the preferred way. Thanks for the comments! -- Martijn Jacobs Four Digits, internet solutions e-mail: [EMAIL PROTECTED] | web: http://www.fourdigits.nl tel: +31 (0)26 44 22 700 | fax: +31 (0)84 22 06 117 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Folderish or SimpleItem object types for structural content
Hello Dieter. This must be a funny classification scheme... Well, I wanted to have the discussion to be general about this thing because I can imagine that this issue has been some point of discussion before, among others, but I will be more specific to explain : - The structure will have categories within categories. (hierarchical categories) - Each category can have custom properties which can be 'inherited' by the aqcuisition feature of zope. - Within the categories you have "object's". - Each "object" can derive his properties through acquisition, or have their own properties (or override the acquisiton properties) - Each "object" will have "item groups", with their own properties / files / comments / etc. - Each "item group" will contain "items", each item will have properties, but also can have properties derived through acquisition from categories and objects : so these "items" will all have different properties, depending in which category they exist. With this rules I was thinking about the next structure : Category (Object Manager, within other Categories) -> Object (Object Manager) -> Item Group (Object Manager) -> Item (Simple Item) But anyway: I have a completely different strategy for you: lets see whether you will like it. You do not materialize the classification scheme at all but you have only your leaves (let's call them "object"s). Each object has its classification as an attribute, say "category". A "category" thus has the form "topic/subtopic/subsubtopc/...". You index the "category" with a PathIndex (I would recommend my "Managable PathIndex") and use canned searches (so called "topic"s) the recreate the classification structure via searches (if needed). If you do the last step (use "topic"s for the complete recreation of the classification structure), then the number of objects will not decrease (what formerly was a classification folder is now a "topic"). But the new structure is much more flexible. You can now assign different categories (say along different dimensions) to your objects and will get different hierarchical (topical) views on your object set. I have been thinking about this kind of structure to use for this specific project. But there is a problem that occures (too) often (in my experience) with this approach. By doing this you really are depending on the ZCatalog : If some conflict errors occure or for some reason the objects are not indexed (correctly) or not updated, some important information is not available for the user. I have experienced alot of problems with unindexed objects, or not reindexed objects due to 'random' conflict errors'. By making a complete hierarchical structure using 'Object Managers', you can always assure that data is accessible, and if the ZCatalog is not up to date, only the search results will not represent the actual structure. 30.000 is not yet very impressing. That's good to know. It's hard to say offcourse, but what is in some way a 'limit' of the number of objects, for instance, if they all have to be indexed? Indexing objects uses ALOT of CPU time for example... In which amount of objects should you reconsider your design? (Speaking of a 'general' guideline) You should take care to use the correct folder implementation, however. As soon as a folder is likely to have more than a few dozen of objects, you should use a "BTreeFolder2" rather than a normal folder. I've used the BTreeFolder2 before for a single folder containing large number of objects, and indeed, it's really boosting the perfomance a lot... Well thanks again for your thoughts, it helps in breaking patterns I'm used too, let's reconsider it using some external input :) Kind regards, Martijn. -- Martijn Jacobs Four Digits, internet solutions e-mail: [EMAIL PROTECTED] | web: http://www.fourdigits.nl tel: +31 (0)26 44 22 700 | fax: +31 (0)84 22 06 117 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
