[Zope] Non-management actions recorded in Undo Log
Hi, we've recently seen a the Data.fs in a few of our Zope systems grow rather fat (from stable sizes around the 100-300MB level to >3GB in the most extreme cases). On further investigation it appears that regular user page requests are being recorded in the Zope undo log - these are not object modifications that have changed the objects themselves (the modification date against all these objects is unchanged), but normal HTTP GET requests. Systems are all 2.11.0 Zope clients with Data.fs served by matching versions of Zeo. Not all of our 2.11.0 apps that are showing this behaviour, only a couple, so I doubt it's something wrong with Zope or ZEO. I'm not a Zope developer, but can anyone shed any light on the possible issues that would cause regular requests to be logged in this way? Regards John ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Centralized sessions with ZEO
You're nearly there! 1. Change the mount point to be something like /session_folder 2. Mount the data fs and create the mount point 3. Create a new transient object container in /session_folder 4. Change /session_data_manager to point to /session_folder/your_new_transient_object_container Restart your zope servers and they should all start to use the same shared session back end. -John --- Unix & Web Infrastructure Management Faculty of Medical Sciences Computing University of Newcastle Email : [EMAIL PROTECTED] Web: http://www.ncl.ac.uk/medev >-Original Message- >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of >Sours, Kevin >Sent: 10 March 2008 21:55 >To: zope@zope.org >Subject: [Zope] Centralized sessions with ZEO > > >I apologize if this has been asked ad nauseam. I've uncovered enough >information online to suggest that it has, but too little to get any >kind of definitive answer. My problem is that I am running a number of >zope instances against a zeo instance. I would like to store the >session data in the ZEO instance so that the session data is shared >between the different zope instances. The question is how best to do >it. > >What I've settled on is the approach given at >http://www.zopelabs.com/cookbook/1061234337. This appears to work fine, >however I don't really like the idea of using an undoable store and >having to constantly pack it. I definitely don't need persistent data >(if the sessions go away when I restart ZEO so be it). I tried the same >approach using a tempstorage instead of filestorage. This works up >until ZEO is stopped and started, at which point the transient object >container in the temporary storage disappears (which makes sense, but is >a little inconvenient). > >I also tried the approach at >http://longsleep.org/1/howto/sharesessionwithzeo. This is absolutely >perfect for what I want to do save for the fact that it doesn't work. >No matter what I tried with it, the sessions don't get shared. In fact, >it doesn't look like the database on ZEO get recognized at all. When I >go to add a mount point it tells me that there is something in the way >of /temp_folder (presumably the autocreated temporary folder object for >the default sessions). What I find particularly odd is that this >appears to be the case for anything configured to mount at /temp_folder >- including the temporary storage configured as part of the default >zope.conf file. Changing that part of the default config or removing it >entirely doesn't appear to have any impact on how zope runs. > >I feel like I'm missing something. It seems like configuring the >sessions to use tempstorage on the ZEO instance shouldn't be this hard. >I'm also confused by the seemingly spurious block in the zope.conf: > > ># Temporary storage database (for sessions) > > name temporary storage for sessioning > >mount-point /temp_folder >container-class Products.TemporaryFolder.TemporaryContainer > > >I haven't been able to find much documentation on what the various >options for this block mean (particularly the container-class) member. > >I should probably mention that I am running 2.8.7, which I realize is >more than a little out of date at this point. I'm hoping that somebody >can fill me in a little or point me to some documentation I may have >missed. > >Thanks in advance >Kevin > >___ >Zope maillist - Zope@zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] apache rewrite quit working
What did `netstat -an` say about your listening ports? John Snowdon - IT Support Specialist -==- School of Medical Education Development Faculty of Medical Sciences Computing University of Newcastle Email : [EMAIL PROTECTED] >-Original Message- >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On >Behalf Of Thomas Bennett >Sent: 25 April 2007 22:44 >To: zope@zope.org >Subject: Re: [Zope] apache rewrite quit working > > >Don't think that is it because it works fine if I change Zope >to port 80. >And, I was accessing apache on port 80 using the ip number, >localhost, the >machine's real name, and the URL domain it just wouldn't >rewrite to 8086. >nslookup showed name was still set to my ip number. >Thomas > > >On Wednesday 25 April 2007 15:50, Jonathan wrote: >> - Original Message - >> From: "Thomas Bennett" <[EMAIL PROTECTED]> >> To: >> Sent: Wednesday, April 25, 2007 4:15 PM >> Subject: Re: [Zope] apache rewrite quit working >> >> > web-dav server is listening on 9800 >> > http server is now listening on 80 >> > http was listening on 8086 until rewrite quit working, >explained below. >> > >> > Thanks anyway, >> >> Any chance your nameserver settings got changed? >> >> Jonathan > >-- > >Thomas McMillan Grant Bennett Appalachian State University >Computer Consultant IIIP O Box 32026 >University Library Boone, North >Carolina 28608 >(828) 262 6587 > >If it's not as simple as possible to try it, then the barrier >to entry is too >high. > >Library Systems Help Desk: http://www.library.appstate.edu/help/ > >___ >Zope maillist - Zope@zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope at universities?
Andreas, We use Zope extensively here in the Medical Faculty, at the University of Newcastle - mainly for Virtual Learning Environments (our MB BS degree program depends heavily on a Zope based VLE for timetabling, resource materials, comms & group, news & bulletins etc.), Electronic Portfolios, Teaching and Learning aids and hosting Collaborative & Research sites with partner institutions and Hospital Trusts. At the last count, we had something like 50-60 separate Zope applications running. John Snowdon - IT Support Specialist -==- School of Medical Education Development Faculty of Medical Sciences Computing University of Newcastle Email : [EMAIL PROTECTED] >-Original Message- >From: [EMAIL PROTECTED] >[mailto:[EMAIL PROTECTED] On >Behalf Of Andreas R. Johnsen >Sent: 17 September 2006 20:45 >To: zope@zope.org >Subject: [Zope] Zope at universities? > > >Hei, > >I'm looking for information about the use of Zope at >universities. Most >large universities have a very heterogeneous infrastructure >and I guess many >of them utilize Zope in a way or another. I'm looking for >references to >universities where Zope is part of a central strategi or in >other ways plays >an important role for several of the web sites at the university. > >Two examples are ETH Zürich and University of Bristol. > >More examples? > > >Regards, > >Andreas > > >___ >Zope maillist - Zope@zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope + Apache on Quad Debian machine
Why do you want to do this? Do you believe that this is more effective or efficient than the scheduling algorithm that Linux uses? Binding an application to a single processor makes very little sense at all. If Zope really is your limiting factor, then you need to spread that load over as many processors as possible, not lock it down to a single cpu - if you do that, then the rest of the processors sit idle while waiting for Zope to respond. Read up on ZEO. Then set yourself a ZEO server up and a couple of ZEO clients, balance your requests between the two using pound, apache or squid. I can guarantee you that if your system is Zope limited, then this will greatly increase responsiveness and the number of requests you can process simultaneously - I run over a dozen dual processor machines hosting approx 60 Zope applications across them, all using ZEO - an average of 5 or 6 instances per system, we have no problem with Zope only using one processor; the applications that are across most machines (up to 10 in our largest application) are very, very fast indeed and can cope with all the concurrent users we can throw at it. Your problem is not Zope itself... you definitely need to investigate further before coming to the conclusion that binding a Zope instance to a single processor is a solution - if you'll excuse me, that's a rather simplistic view to take. I was also suggest reading up on Apache, Linux, Proxying/Caching, Clustering and Load Balancing - they're all factors to consider when developing and deploying a web application. John Snowdon - IT Support Specialist -==- School of Medical Education Development Faculty of Medical Sciences Computing University of Newcastle Email : [EMAIL PROTECTED] >-Original Message- >From: [EMAIL PROTECTED] >[mailto:[EMAIL PROTECTED] On >Behalf Of Hugo Ramos >Sent: 15 March 2006 20:45 >To: [Zope] >Subject: [Zope] Zope + Apache on Quad Debian machine > > >Yellow, > >I'm using Zope+Apache on a 4 xeon's/4GB ram machine running Debian. >I've noticed that the CPU's never go beyond 30% top occupation... but >on rush hours the site takes too long to load... > >I've been reading about process affinity and how it could speed up >everything by making zope run on 1 CPU, Apache on another and so on... > >Has any1 tried this before? Can you point me to some documentation? >What's your experience? is it true that not doing this the 4 cpu's >will not be 100% used? > > >Cheers >-- >Hugo Ramos - [EMAIL PROTECTED] >http://www.orkut.com/Profile.aspx?uid=10082105466310142690 >___ >Zope maillist - Zope@zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope Scalability
>-Original Message- >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On >Behalf Of Jens Vagelpohl >Sent: 07 October 2005 09:04 >To: Zope ML >Subject: Re: [Zope] Zope Scalability > > >> As an aside, we find management of ZEO clients much easier >if each ZEO >> client of a particular system shares the same products and external >> methods via an NFS share. That way we can untar one product and >> push it >> out to all of the clients simultaneously. > >I'd be a little afraid of creating a single point of failure with >NFS. I have used setups like that before, but personally prefer some >simple distribution mechanism instead. > >If you use CVS or SVN for your software you could write simple SSH >scripts to visit hosts and do a cvs/svn up in the right place and >then restart the clients. rsync is a good candidate as well. > >jens > We don't have anything that complex :-) Most sites that we host probably have less than 20 external python methods (a lot have none at all!), with common products shared by all the sites installed on each system in the master zope lib/python/products directory structure. The majority of complexity is in each systems ZODB. All of our boxes are on a private network, on the same gbit switch as the ZEO server, so performance/service interruption, network timeouts etc.. is never a factor. The NFS shares (products, extensions, file upload areas[uploads via zope, and these areas are then served back out via Apache] and user home directories[again, using our ldap systems for authentication]) are from the same ZEO server (primary storage is handled by a multi-terabyte SATA). We're running about ~50 individual Zope systems at the moment (all as ZEO clients, many have multiple ZEO clients across several nodes, with incoming requests load balanced via Pound).. and I must say, this setup is way better than our old Sun E3500 that used to (try) to handle all of this! ZEO is the way to go for sure though we are now hitting problems when trying to integrate the Internet2's Shibboleth single sign on authentication system into a load-balanced setup... but that's a story for another time :-) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Zope Scalability
>--On 5. Oktober 2005 17:37:08 +0100 Tom Wilde ><[EMAIL PROTECTED]> >wrote: > >> Hi folks, >> >> I've been using Zope for a while now and found it to be an excellent >> solution however I've got a few enterprise questions about the >> scalability of Zope that just need to be answered. >> >> [ nb: I'm using V2.72 at the moment, not having moved over to V3 ] >> >> ExtFile >> Managed to serve 500,000 files through extfile - anybody >come across an >> upper limit here? Planning to serve larger sites off a SAN >hopefully - >> has anyone tried this? > >no idea For static content we still use Apache. >> >> Zeo and Cataloging >> If I have my site on one server and ZCatalog (using TextIndexNG) on >> another server (connected by ZEO) where does the actual >index operation >> occur? Do just the results get propogated to the catalog? > >The indexing work is always happens on the ZEO clients, not on >the server. >Just the results are transferred to the ZEO Server. > >> >> LDAP Authentication >> I understand that can use the LDAPUserFolder Product to provide >> authentication against an ldap server. Could I throw,say, >5+ users >> at zope using this system? > >This should work in general. Possibly it requires some >tweaking here and >there depending on your usecases. We use LDAPUserFolder as the backend authentication store for our online learning environment here at newcastle. It copes rather well with several thousand students and many groups - I'd imagine it will scale as well as your LDAP setup can. (Our LDAP server is a pretty low-end Sun V60 - single Xeon 2.8GHz with 2Gb of ram and is very rarely under any significant load). >> >> Load Balancing and Cookie Based Sessions >> Currently we use cookie based sessions via cookie crumbler >presumably in >> a load balanced server setup we'd have to host acl_users ona ZEO'd >> izope instance somewhere on the backend? > > >I don't see what sessions have to do with acl_users. In your >ZEO setup you >have identical data except sessions on every ZEO client. You >just have to >ensure somehow that users get to the same ZEO client for the >lifetime of >the session. How this is done depends basically on the load balancer. We initially tied users sessions to individual ZEO clients (6 x dual Opteron nodes) and it worked well. However we have since migrated the session information to be served via the ZEO server in the same manner as the data.fs; we now get the benefit of much finer grained load balancing and are very pleased with the performance we get. As a side effect, it means we can host multiple ZEO clients from different systems on one box without fear of a particular ZEO client bringing the box to it's knees. As an aside, we find management of ZEO clients much easier if each ZEO client of a particular system shares the same products and external methods via an NFS share. That way we can untar one product and push it out to all of the clients simultaneously. -John -- Faculty of Medical Sciences Computing University of Newcastle upon Tyne ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] cpu performance zope/zeo
We're using dual cpu (not dual core) 64bit AMD Opteron (Sun Fire V20's) based systems and are VERY impressed with their performance; our python benchmarks show dual Opteron 250 systems outperforming dual twin-core Xeon 3.6GHz with the same code. Almost everything we run is ZEO based and hosted over 2 or more boxes (7 or 8 in the case of our biggest system). We couldn't be happier with the combination of Opteron and ZEO. We benchmarked 64bit Opteron, 64Bit UltraSparc III, 32bit Xeon and 64bit Xeon for our applications. Opteron trounces them all in day to day use. Sun's dual core Opteron offerings are coming out VERY soon. So that will make the V20 a 4-way system, and the V40 an 8-way system! As a guide, our standard spec for a ZEO hosted Zope server is 2 x Opteron 250/252, 4Gb, dual Gbit NIC, dual 10krpm SCSI drives in RAID1 (the main ZEO backend server is a similar spec but utilising a big SATA array in RAID50). We've found we can host 6 or so big sites (resident seize of 250mb or more) on each node and still get excellent performance by duplicating across several nodes. We have ZEO served sessions and use Pound to load balance across the pool without any stickyness - meaning on each page access users will (potentially) go to a new box - resulting in much finer grained balancing and use of resources. Performance is *excellent*. -John John Snowdon - IT Support Specialist -==- School of Medical Education Development Faculty of Medical Sciences Computing University of Newcastle >-Original Message- >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On >Behalf Of Srdan Zagorac >Sent: 26 July 2005 06:27 >To: zope@zope.org >Subject: [Zope] cpu performance zope/zeo > > >Hi > >I need a suggestion / advice regarding the CPU options for the best >ZOPE and ZEO performance. eg 32 vs 64 bits architecture, >single vs dual >core, hyper threading. >Is there any bechmarks already posted on the web?? > >Thanks > >-- >- >Srdan Zagorac >Software Developer / Webmaster / DB Administrator >Bioengineering Institute >University of Auckland >email: [EMAIL PROTECTED] >url: http://www.bioeng.auckland.ac.nz >ph: +64 9 373 7599 ext. 82046 >fax: +64 9 367 7157 >- > >___ >Zope maillist - Zope@zope.org >http://mail.zope.org/mailman/listinfo/zope >** No cross posts or HTML encoding! ** >(Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
