Re: [ZODB-Dev] grab zodb dict keys by search term
ZODB has nothing built-in. You build it on top of the ZODB. I think what you are looking for is something like Hypatia - https://github.com/Pylons/hypatia On Tue, Dec 10, 2013 at 4:46 PM, Tamer Higazi wrote: > Hi people! > > I am working a lot with IOB and OOB Trees as well with PersistentDict and > PersistentList to store my Data in ZODB. > I want to ask if there is a way to grab data from a dataset by search time. > > I know only the SQL way: > > select lower(COL1) from table1 where COL1 LIKE 'aa%' > > so usually I want to get the entryies only that starts with 'aa' > > do I have to iterate the entire dict with an interator or are there builtin > functions that could accomplish those tasks for me ?! > > > I would kindly thank you > > > > Tamer > ___ > For more information about ZODB, see http://zodb.org/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > https://mail.zope.org/mailman/listinfo/zodb-dev -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] polite advice request
> > Would you implement a column store, and how would you do that? > > Ditto. So many Dittos, it sounds like a Rush Limbaugh talk show :) > "large" can mean many things. The examples you give don't > seem very large in terms of storage, at least not for ZODB. One app we have is 26,344,368 objects. ZODB is the least of its concerns. > It's really hard to make specific recommendations without > knowing more about the problem. (And it's likely that someone > wouldn't be able to spend the time necessary to learn more > about the problem without a stake in it. IOW, don't assume I'll > read a much longer post getting into details. :) This is fair. ZODB is intimately tied to the application design so it is a bit difficult for someone to qualify what they are doing without having to explain the application design. This sucks from a newbie's point of view but its reality. I just wrote up some thoughts on ZODB. Might be useful for others - doubtful - but maybe. https://docs.google.com/document/d/12RGOTSMrl0CttkCZJ5rp-TSaakAY2Pn4VnWhVMcFMQw/edit?usp=sharing Anyway. Tismer if you write up more thoughts; I will read them. Not guaranteeing a response. cheers -- Alan Runyan ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] make ZODB as small and compact as expected
Pretty sure this could be merged to zopefoundation project if anyone asked Carlos. On Tue, Jul 23, 2013 at 7:47 AM, Adam GROSZER wrote: > On 07/22/2013 01:27 PM, Jim Fulton wrote: > >> >> - The biggest thing ZODB needs right now is documentation. >>Unfortunately, this isn't easy. There is zodb.org, >>but much better documentation is needed. >> > > There is > > https://github.com/cguardia/**ZODB-Documentation<https://github.com/cguardia/ZODB-Documentation> > > but seems like it got stalled 5 months ago. > > > -- > Best regards, > Adam GROSZER > -- > Quote of the day: > Each time you are honest and conduct yourself with honesty, a success > force will drive you toward greater success. Each time you lie, even with > a little white lie, there are strong forces pushing you toward failure. > (Joseph Sugarman) > > __**_ > For more information about ZODB, see http://zodb.org/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > https://mail.zope.org/mailman/**listinfo/zodb-dev<https://mail.zope.org/mailman/listinfo/zodb-dev> > -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Broken objects problem.
> Thanks for your response , i check the imports and seems to be ok but > still not working. print sys.path from inside flask app and from the console script you run which *does* work. then compare both of these sys.path listings. -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RFC: release persistent as a standalone package
> I would like to release a '4.0.0' version of the package, and switch > the ZODB trunk to pull it in as a dependency (deleting the currently > included (older) copy of persistent). One possible issue is that I > have not (yet) made the C extensions work under Python 3.2: I don't > know whether that should be a blocker for a release. I do not believe its a blocker. That is a "feature" that could be added. > Comments? You rock. -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zeo server hot failover
> So an error is raised and logged, but with the cache being cleared so that > on the second try it reconnects? My rational for this change is that If > your doing a hot failover that means that a) something bad has happened to > main server and the recovery of those transactions probably won't happen any > way or b) it happened during a maintenance window/the failover is happening > for convenience and any difference between the two servers are probably > minor, such as session data. With b. the server admin would be in the > position to restore the main server anyways. Sounds reasonable. Does the "drop rather than verify" feature get verified when connecting to a new replica? Should it? -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] A little help in configuring a better solution for ZEO deployment
> I use 4 processes with 7 threads each for serving data Any reason for 7 threads? Try 1 thread? Is it a classical web application? Or you creating threads in your application? > WSGIDaemonProcess doba.ua user=www group=www processes=4 threads=7 > maximum-requests=200 > > I have little space on /tmp partition, so the cache of ZEO is only 40 MB You can change the tmp location in Python by setting TMPDIR environment variable for your process. I would advise against persistent disk cache and increase the ZODB cache-size. If you are using latest ZODB you can use the cache-size-bytes which mostly works. You can set it so zodb cache for each process will not exceed, say, 512MB. > I can't find any ther explanation, that ZEO does not handle the load well > enough. If your ZEO server process is not bottlenecking - it is your application. > Does someone see any way to ease the load? dont know your application but it is doubtful that ZODB is your bottleneck. -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Relstorage and MSQL
> Windows 2008 server standard. Python 2.6/2.7 > I'm not sure. I think living without clustering and memcached for the moment > is fine. I would agree. >> And what about blob storage? Are blobs inside the DB in little chunks >> enough, do you want them on the filesystem via shared network drive or new >> efficient filestream support in SQL server 2008+ > > > I don't know enough to answer this but for our purposes support SQL Server > in a basic capacity so Plone runs ok is enough to start with. This really depends on usage patterns and capacity. I would recommend keeping the BLOBs on the filesystem. > Our need is to 'state' support for SQL server to our client. They have SQL > server licenses and they'd prefer to use them. The job is for 3 plone sites, > public, extranet and intranet but with no estimate yet of the data size. > Availability concerns are an issue. > In the case where the work goes ahead and if performance is an issue then > we;d be in a better position to either implement greater support for > sqlserver or recommend they switch to a more mature relstorage option. Hope > that helps put things in context. If you can depend on mxODBC then I do not believe this is a huge problem. I would be skeptical of pyodbc stability/performance - its worked ok for us but we dont have anything 24x7 running with it. As someone said earlier it is not difficult just very time consuming to test. I would be more than happy to help test. Some more thoughts: - If you run mxODBC you will have much less adoption/testing by the community. due to license and pain to install mxODBC. - pyodbc i would skeptical of and test. you would get the most usage using this driver. - pywin32 is another candidate. it should work just fine but you may need to watch for scaling issues (you may have to add some smarts the mssql storage) I would love to see this support added. -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] How to update an object in a multithreading application?
> I am updating an item in a separated thread but when it finishes the new > value is not updated in the same object in the main thread. You need to call sync() on the connection object to see changes from another thread. http://zodb.readthedocs.org/en/latest/api.html#connections > In the example below root['counter'] starts in 0, is incremented in the new > thread but it remains in 0 in the main thread. if your increment a counter its best the counter be of type, BTrees.Length cheers, alan ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] where does 32it vs 64bit matter?
On Wed, Dec 21, 2011 at 12:35 PM, Chris Withers wrote: > Hi All, > > I have a set of ZODBs, they're currently served from a 64-bit mysql > cluster running relstorage. > > At the moment, all the clients are 32-bit. At some point, those may be > changing to be 64-bit. > > What are the implications of this? > Do all relstorage clients need to be the same bit width? > What about FileStorage and ZEO connections? > > Are there any migration steps or minimum ZODB/RelStorage versions needed > to make this step? We run 32bit windows clients against 64bit linux zeo server; works fine. > -- Alan Runyan Skype/Twitter:: runyaga Office:: 713.942.2377 ext 111 http://ploud.com/ Plone site in less than 10 seconds ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB on ironPython?!
On Thu, Oct 20, 2011 at 12:39 PM, Tamer Higazi wrote: > Hi! > I am interisted getting ZODB to run on ironpython. Is that at the moment > possible?! > > As I red, all python modules could be loaded on .NET without any > problems as long they wouldn't need any compiled modules. Is that at the > moment possible running ZODB on .NET ironpython?! > Not until a pure Python implementation of persistent and BTree's exist. There has been substantial amount of work done but it more is required. runyaga ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RFC: Blobs in S3
On Wed, Jul 6, 2011 at 1:44 PM, Jim Fulton wrote: > We're evaluating AWS for some of our applications and I'm thinking of adding > some options to support using S3 to store Blobs: Mind clarifying, is the application on-premise but Blobs are in AWS/S3? Or your application running in AWS/EC2 and storing Blobs in S3? alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Advice on whether to run relstorage database and Zope on different servers
> We've been successfully running Plone 3.3 + Zope 2.10 + Relstorage + MySql > for a couple of years now. We're running low on memory on our existing > server (which has numerous other services on it) and will soon need to deploy > an additional Plone 4.1 + Zope 2.13 + Relstorage + Mysql environment. We've > been using virtual environments since September. > > For the purposes of this discussion please assume we don't have the option of > increasing the memory on our existing server and so need to add a new > one.Some questions I'd really appreciate your help with please: > > * I'm familiar with enterprise environments where you would have an "app > server" and a "database server" but are there any advantages to putting Zope > and MySQL on different servers? > * I'd expect a performance hit if we run Zope + MySQL on separate servers but > is this hit manageable? > * Can we put anything in place to mitigate this? > * Would you simply recommend having the existing mysql + zope on one server > and the new mysql + zope on the other? Anthony this isnt a ZODB question; its a plone related application question. Ask plone-users. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
> Right, but, even if the MySQL server is 64-bit, if the clients are > 32-bit, they won't have this problem, will they? if your running a 64bit python - you will see substantial memory growth. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage support to Microsoft SQLServer
> I have read that there is a problem to implement MS-SQL adapter for > Relstorage because the “Two phase commit” feature is not exposed by > MS-SQL server . unsure about that. probably depends on the client access library. > Is there solution to overcome this problem, Without introducing too many > layers? > Can we use PyMSSQL and ADODB Python extension to implement the > relstorage Adapter for MS-SQL. i recently had a discussion with some guys about this. i am unsure what their analysis was. but my opinion: - adodbapi is not good. - pymssql i've not used - pyodbc we used but it doesnt support storedprocs. works ok. - mxodbc we use and highly recommend. yes mxodbc costs money but you have support. i spoke with shane about this in the past about which library would he probably use if he were to support mssqlserver and his unresearched/not definitive answer was mxodbc. mainly because its supported and has been in production usage for almost a decade. > Has any one tried this before? no. > Are there any other major concerns you would have for developing MS-SQL > adapter component? shane would be best to answer this. > Please point us in right direction on the subject. many of our customers use mssqlserver; we would be very interested in the outcome. so please keep the zodb mailing list up-to-date with your progress. my suggestion: use mxodbc. involve shane so he can fold it into the main relstorage line in the future. maybe even budget some money to pay him so he has interest in the outcome of the mssql/odbc adapter for relstorage. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Book Update
>> Thanks to Carlos and Jens we have the ZODB book being updated >> daily on zodb.org, http://www.zodb.org/zodbbook/ -- also have >> disqus integrated into sphinx html output. People can read the >> first three chapters and provide feedback at the bottom of each page. > > Posting comments is a pain. I have to type in my contact info each time. really? could be a bug in disqus? are you using disqus, fb, twitter, openid? which oauth provider are you using? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB Book Update
Thanks to Carlos and Jens we have the ZODB book being updated daily on zodb.org, http://www.zodb.org/zodbbook/ -- also have disqus integrated into sphinx html output. People can read the first three chapters and provide feedback at the bottom of each page. We are looking for feedback. If anyone is interested in contributing; maybe starting a recipe-style section for the book. We can get you account and you can start contributing on github (svn is supported). cheers alan runyan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Can RelStorage be used as a ZEO Storage back-end?
> I'm starting to dig into the code, but perhaps this can > be answered quickly: can RelStorage be used in place of > FileStorage for ZEO storage? My understanding is Yes. Try it. IIRC you can even use a ZEO storage in place of a FileStorage. You may want to specify what version of python, zodb you are using. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB replication and high-availability across data centers
> we are currently investigating options for a large-scale solution > where the machines (ZEO clients and ZEO server) should be separated > across two data centers. Incoming traffic should be balanced between > both data centers. Automatic failover and online-replication are > desirable - "high-performance" is a must. > > The well-known options are: > > ZRS > (ZEO) > Relstorage on top of some RDBMS-related clustering/replication mechanism > NEO > DRDB > ZEORaid We have not explicitly set something like this up; although our hosting partner has SAN replication between data centers. This is another solution; while low level - may be valid to explore. They use it with Oracle and MSSQLServer to do cross datacenter replication. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
>> I went for the following in the end: >> >> ... >> cache-size 200Mb >> client app >> wait on >> >> mount-point / >> cache-size 5 >> cache-size-bytes 500Mb >> >> Any obvious issues with that? > > No. each zeoclient needs a name in your conf. required so the .zec files dont collide. this is what is happening to you. look at component.xml again for details on assigning zeo client a name or identifier. that will give the filename a suffix like something-$identifier.zec please open issue at launchpad; if this fails zope should NOT start. alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
> I'm wondering if some of my issues might be alleviated by persistent > client caches. anything is possible. > Now, I have the impression that these were unreliable and buggy and had > been so for some time. they appear to be more stable now. we use them on several customer's in very specific cases. mainly with infrequently modified databases. > Is this still the case? If not, where are the docs on how to configure > them nowadays? http://svn.zope.org/ZODB/tags/3.9.7/src/ZODB/component.xml?rev=117035&view=auto Enables persistent cache files. The string passed here is used to construct the cache filenames. If it is not specified, the client creates a temporary cache that will only be used by the current object. -alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Windows/ZODB Tests Passing
For anyone who cares. ZODB trunk builds win32 and amd64 have been passing tests for almost a week without failures. Thanks Jim for getting ZODB tests working reliably on Windows! cheers alan p.s. let us know if you need py2.7 builders and what to test. you can find all the builders at hudson.enfoldsystems.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] read-only database
> We're currently deploying plone with zrs which replicates to read-only > databases which is what actually gets served to the user. It works > fine in most cases, but there seem to be odd issues here and there > where plone will try to write to the database and will throw an error > on the page load. We have read only Plone's in production and this works fine. Please enumerate the problems on a plone ticket, dev.plone.org (mention your py, zope, plone versions). > Is there anyway to get it to not error out and not simply silently not > write to the database? It would be much safer to fix Zope/Plone application than to monkey around with silently working around the readonly aspects of the database. especially since you are already putting the database into a readonly mode. again, we are doing this fine in production. so it must be something in your application causing writes. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB Book Project
http://zodbdocs.blogspot.com/ - ZODB Book Project Hello ZODB-dev, I wanted to provide a status update for the upcoming ZODB book. We have achieved our initial financial goal to get an advance to Carlos de la Guardia. He will begin planning and writing. We need more money. Please take this opportunity to goto the blog, review the book outline, provide feedback and click on the paypal link and add to the pot. I will be contacting companies who are absent from the donor roster. Please contribute. Both individuals and companies can contribute by becoming authors on the blog and writing articles. I've gotten a fairly luke-warm reception from the community on writing articles. If you have any experience - let me know. Email me an article, I will put it up and credit you. This project will succeed only by pulling the resources together from our communities. http://zodbdocs.blogspot.com/ - ZODB Book Project cheers, alan runyan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] SpatialIndex
> Mixing meta data into names is a bad idea. You don't want to change > the name of something just because some of its meta data changes. Your right. It's just that sometimes people may have a package name that is not agreeable to the broader community. >> The namespace could be, zodb. > > No it can't. ZODB isn't a namespace package and can't be one without > breaking backward compatibility. Should it be a namespace package over the long term? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] SpatialIndex
I have similar feelings as Matthias. My thoughts: - Have a canonical namespace which denotes quality, proven, and widely used packages. Packages should never be in this namespace unless the package can meet some quality level. The namespace could be, zodb. - We could use something similar to the Apache incubation process. - We should come up with a list of criteria which we want this complete package to achieve. I believe everyone feels that the Standalone ZODB package is simply not enough to build a sophisticated Real World application. So we need to outline what are the min. features of such an aggregation. This is a huge project. So I completely understand the hesitation to invest into something so daunting. Maybe the beginning is to blog and talk about the packages that DO work and get confirmation that the authors are still interested in maintaining these packages. > I'm sure the ZODB documentation project > (http://zodbdocs.blogspot.com/) would gladly accept a contribution > towards clarifying what current best practice is for catalogs in ZODB. Yes. If you are interested in sharing what packages you are using in production. Let me know. I will give you access and you can post blog entries on zodbdocs. No one has taken me up on the offer. I would really like to see others contribute with their ZODB experiences. Preferably talking about direct experiences with packaging instead of talking about the LACK of packaging on the blog would be good ;) cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB Docs Blog Open for Contributions
http://zodbdocs.blogspot.com/ Unfortunately we could not get a 501c3 organization to accept donations for the project in a timely manner. Enfold Systems has wired up is paypal account and will take donations on behalf of the ZODB book. Once we get the 2k in donations; we will transfer the money to Carlos and the writing will begin. Carlos is expected to have a revised draft of material to be covered by Monday evening. If you are interested in contributing some writing / materials around ZODB. Please let me know. I can give anyone contributor access to the ZODB Docs blog so that interested parties can post recipes, howtos and thoughts around the ZODB. Carlos is open to have any submissions be integrated into the upcoming ZODB Docs book! We are open for business. Please contribute. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Book
> Greate idea! Just, what happens if the goal isn't met? > Plus, I'd really like to see docs on DirectoryStorage. If I can not get enough contributions for an initial advance for Carlos to begin work -- the project will be aborted. As we start getting contributions there will be more activity on the outline / material expected to be covered. alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Book
> How to sign up as a volunteer? I have you signed up to volunteer- thanks! alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB Book
Hello ZODB Community, I wanted to announce a community-led initiative to create a ZODB book. The goal is to solicit contributions from the community and when we reach a threshold, Carlos de la Guardia will begin work on writing the book. I have started a blog, http://zodbdocs.blogspot.com/ The blog contains the contribution levels and the proposed outline. Any feedback is appreciated. If you are interested in contributing financially please subscribe to the email list on the blog. I am working on getting a paypal account. I will keep the blog up to date and will ping the mailing list when we start accepting contributions. cheers alan runyan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?
>> I suppose you could look at zeoraid. > > Hadn't thought of that, will look at it, thanks! Anyone using ZEORaid in production? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Simple RelStorage Question
> Does RelStorage handle concurrent writes correctly? This is probably > so obvious as to ask if the sky is blue, but I couldn't tell from the > documentation if I could setup multiple Zope (Plone in my case) > instances all using RelStorage to talk to the same database without > incident. relstorage handles concurrent writes. It is still possible to get ConflictError's. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Ever-Increasing Memory Usage (even with cache-size-bytes)
On Tue, May 18, 2010 at 1:14 PM, Ryan Noon wrote:
> Hi All,
> I converted my code to use LOBTrees holding LLTreeSets and it sticks to the
> memory bounds and performs admirably throughout the whole process.
> Unfortunately opening the database afterwards seems to be really really
> slow. Here's what I'm doing:
> from ZODB.FileStorage import FileStorage
> from ZODB.DB import DB
> storage = FileStorage('attempt3_wordid_to_docset',pack_keep_old=False)
> I think the file in question is about 7 GB in size. It's using 100 percent
> of a core and I've never seen it get past the FileStorage object creation.
> Is there something I'm doing wrong when I initially fill this storage that
> makes it so hard to index, or is there something wrong with the way I'm
> creating the new FileStorage?
Is there a 'index' file that is being created? It would be in the
same directory as the database file.
How are you closing the application?
If you see the index file changing when you start up; it is probably
rebuilding the index.
-alan
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Ever-Increasing Memory Usage (even with cache-size-bytes)
> So, the best place to call those would be during my commit break (whenever I > decide to take it? [which would be less often if I could be sure of no > crashing]). Are there any other problems with the way I was using ZODB in > my code? I really like it, but I recognize that it's a lot more complicated > than my old system. Correct. Pick appropriate place where you are "finished" with a batch of objects and possibly have them no longer referenced. Then you can call cacheMinimize. You can call cacheGC to reduce memory usage. Your code looks straight forward. I do not see it doing anything strange. It's more complicated than old system? Ugh. That sucks. The ZODB should be LESS complicated than sqlite+custom rolled caching. If not than you may be doing something wrong or ZODB is not living up to its promise *wink* cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Ever-Increasing Memory Usage (even with cache-size-bytes)
> The DB on the choked process is perfectly good up to the last commit when it
> choked, and I've even tried extremely small values of cache_size_bytes and
> cache_size, just to see if I can get it to stop allocating memory and
> nothing seems to work. I've also used string values ('128mb') for
> cache-size-bytes, etc.
On the connection object there are two methods you want to use:
- cacheMinimize
This is more of a heavy hand which attempts to deactive *all* non
modified objects from cache.
- cacheGC
This will clean up the internal cache via the cache-byte-size parameter.
If you are not calling these (I do not believe they are called in trnx.commit)
in your code; then they are probably not being called.
cheers
alan
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Author ZoDB Books - Packt Publishing.
I think a ZODB book would be a wonderful edition to Packt's NoSQL library. I really believe the ZODB book should *not* talk about any web framework integration. There is plenty of information to convey about ZODB without talking about integrating it into external projects. It would be great if the book, at some later date, would have opportunity to be released under CC license. The ZODB project needs external facing documentation. I just got back from a conference where I met one of the MongoDB guys. Obviously mongo is a different animal. The features I saw that were interesting: - Indexing/Querying support. - Ability to auto - (in/de)crement counters on records. - Language independent bindings for MongoDB If a ZODB book is written I would highly suggest detailing a indexing/querying strategy Even if it is use zope.catalog/zope.index with some examples. fwiw, alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB 3.9.5 (was Re: "cache-size-bytes" problem)
> > There are a few issues that seem to have patches, will those make it? > > - https://bugs.launchpad.net/zodb/+bug/544305 > > This will be in. Are there others? Nope. I thought another patch was sent but we need more thinking before suggesting this patch. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] "cache-size-bytes" problem
> Hello, > > > > So far I know Hanno fixed that. > > Any chance to get it released as 3.9.5? > > Absolutely, it will be released in 3.9.5. > > This will likely happen in 2 or 3 weeks. > Good to hear. There are a few issues that seem to have patches, will those make it? - https://bugs.launchpad.net/zodb/+bug/544305 I think we can close this: - https://bugs.launchpad.net/zodb/+bug/114814 There are other "low hanging" fixes. Should someone create a patch? e.g. https://bugs.launchpad.net/zodb/+bug/239086 alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Splitting a large ZODB file storage - any light-weight approach?
> we have some huge ZODB file storage (60-80 GB each) and we want > to split the storage into a number of smaller storage (easier > to handle for backup and packing). The storages themselves only > contain a few (Zope 2) folders - let's say folders A, B, C - > within the root and we want to create seperate storages for A, B, C. > > Is there some easier approach than using the Zope 2 CopyManager API > for moving data around. The ZODB storage iterator API comes to my mind... > is this a suitable approach or is there some other light-weight approach > (compared to using CopyManager API)? > ExportImport module? http://svn.zope.org/ZODB/trunk/src/ZODB/ExportImport.py At one point I suggested a feature for Zope 2 which would be "Export as Mounted Storage". Which could generate a database with your exported content in it. So you could simply wire up the new storage in .conf file, delete the original and remount it with ZMI. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Data.fs size grows non-stop
> I'd just like to add that there's some changes that can be related to this: > > - we had some classes inheriting from Persistent that now inherit from > something else as well (but no extra arguments are being added, AFAIK); > - we added some zope.interface definitions to some Persistent classes; > > maybe this causes some kind of behavior that we were not aware of? I doubt it. Thousnads of people are doing this and do not report the same behavior. What is more likely is that a programmer is changing a persistent object very often. One of the downsides of the ZODB is that it is so transparent it is possible to unwillingly make database changes. A design pattern for RDBMS is to have 2 pools. READ pool and WRITE pool. Often the READ pool comes from some replica and WRITE is to the master. I'm unsure this pattern would work for ZODB. I know Malthe was thinking about this but unsure if he had anything concrete. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] [OT] NoSQL
> I am very intrigued now to setup an EC2 cluster and install a z3c.sharding > based solution demonstrating 100M users with some data. Mmmh... That is the great thing about EC2. You can do massive experiments on the cheap. Actually one of our interns is doing some work on ZODB. He is doing mostly narrow calculations on efficiency of ZODB. i.e. how structure of ZODB Filestorage could be changed to better use disk cache, etc. Possibly interesting to the community at large. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] instrumenting a ZEO server?
>> What version of ZODB are you using? > > Whatever ships with Zope 2.9.8 I think that is ZODB 3.6 which is on the older side. I was simply noting that there are several invalidation fixes in ZODB 3.8; whether they fix your problem I do not know. >> If you have zeo on in debugging > > What does this mean? Turn up logging in zeo.conf to DEBUG ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] instrumenting a ZEO server?
On Wed, Aug 26, 2009 at 8:08 AM, Chris Withers wrote:
> Hi All,
>
> I'm wondering if there's any way to get a ZEO server to log about how
> it's performing?
> (eg: how many connections, how many clients are waiting, where they're
> waiting, etc)
>
> I'm trying to debug a problem where app servers are being dropped out of
> a load balancer pool for being unresponsive, and I think they're waiting
> on the ZEO server so I'm trying to get some stats on the ZEO server side
> as to what's going on...
>
> As far as the clients go, DeadlockDebugger is providing some insights:
>
> File "/opt/Zope-2.9/lib/python/ZEO/ClientStorage.py", line 749, in loadEx
> self._lock.acquire() # for atomic processing of invalidations
>
> I assume this is the client waiting for the storage server's lock?
> Is this the ZEO server's single threadedness or is this a lock between
> threads on the app server?
> File "/opt/Zope-2.9/lib/python/ZEO/ClientStorage.py", line 769, in loadEx
> data, tid, ver = self._server.loadEx(oid, version)
> File "/opt/Zope-2.9/lib/python/ZEO/ServerStub.py", line 192, in loadEx
> return self.rpc.call("loadEx", oid, version)
> File "/opt/Zope-2.9/lib/python/ZEO/zrpc/connection.py", line 531, in call
> r_flags, r_args = self.wait(msgid)
> File "/opt/Zope-2.9/lib/python/ZEO/zrpc/connection.py", line 638, in wait
> asyncore.poll(delay, self._singleton)
> File "/usr/local/lib/python2.4/asyncore.py", line 122, in poll
> r, w, e = select.select(r, w, e, timeout)
>
> What does this mean? I'm guessing it's the actual transfer of data from
> the storage server to the app server, right?
What version of ZODB are you using? If you have zeo on in debugging
what you can see is if there are outstanding clients waiting to write
while a transaction is already writing. It says something like "# clients
blocked while transaction in progress" in zeo server log files.
How many connections:
iirc zrpc is connection oriented so you should have # of application clients
connected to ZEO server. Or it may be # of clients * # of ZODB threads.
How many clients are waiting:
You should be able to see this in zeo log files
Where they're waiting:
deadlockdebugger or products.signalstack will show what the clients are doing.
may want to strace the client and server to see if there is long
pauses of inactivity.
maybe your running up against a real ZODB bug that was fixed in later releases?
look at ZODB 3.8 branch NEWS.txt
good luck
alan
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Temporary files not closed
On Thu, Jun 25, 2009 at 7:19 AM, Izak Burger wrote: > Benji York wrote: >> Have you looked at the contents of the files? Once you have an idea of >> what is being written you might be able to determine what is doing the >> writing. > > The big 6.3GB one contains pickles. Lots and lots of pickles. > > I suspect we're simply dealing with a process that creates a LOT of > savepoints before finally committing. The developer of the application > is out at the moment, when he gets back from his lunch-break/touch-rugby > game I shall ask him about this. We have been seeing something similar on Windows. Where there are lots of zeo-cache files + .tbufs lying around. This does not happen all the time. I was presuming it had something to do with signal handling/windows/tempfile. We have not found a way to reproduce the problem but have seen it at several customer environments. -alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] "Transaction blocked waiting for storage"
On Mon, Jun 22, 2009 at 10:06 AM, Andreas Jung wrote: > On 22.06.09 16:56, Pedro Ferreira wrote: >> Hello all, >> We occasionally get some messages like this in the logs: >> >> 2009-06-22T14:12:42 (30651/137.138.128.213:35787) Transaction blocked >> waiting for storage. Clients waiting: 1. >> >> But today we eventually got to: >> >> 2009-06-22T16:00:11 (30651/137.138.4.153:53416) Transaction blocked >> waiting for storage. Clients waiting: 204. >> >> Which lasted some minutes. During that time, our system seemed to hang, >> even if it was possible to connect to the db through the command line. >> It eventually went back to normal by itself. >> Any clue on what might have caused this? > The reason is likely a long running transaction. Since the storage server > is single-threaded, a long running transaction will block pending requests. Question: Will the ZEO server respond to LOAD requests from ZEO clients while WRITEs are pending? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN
On Tue, May 26, 2009 at 1:58 PM, Shane Hathaway wrote: > Jim Fulton wrote: >> FileStorage indexes can't be saved after they reach a certain size, >> where size roughly based on the number of objects. >> >> I need to find a way to fix this. > > It might be interesting to use SQLite for FileStorage indexes. With > SQLite, we wouldn't have to store the whole index as a unit. Or possibly BDB? The indices appear to be fairly straight forward to store. I know people have had nightmares with BDB. We use it in our Enfold Proxy cache and it works fine. We have not had corruption or other issues that tend to come along with BDB usage. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB Documentation
Would it be useful to have a document that starts detailing sections that are missing in ZODB/ZEO programming guide? i.e. - Blobs are not mentioned. - ZEO/Persistent disk cache not mentioned - xdb reference safetybelt - Documentation of various backend storages - Relstorage, Demostorage, Filestorage - MVCC is not mentioned - Persistence cache not described - savepoints vs. nested transaction - Diagnostic recipes (how to diagnosis your application) - Known Limitations How do you guys suggest we work on documentation? Would it be useful to come up with an outline of the target documentation? Then people could start filling in some details. Maybe Jim could then review instead of produce the documentation? cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] analyze.py
Attached is a minimally changed analyze.py file, it adds a min, max size of class types. Maybe someone can help me. We have a customer whose running out of memory. We suspect it is content related, i.e. they uploaded a really big file and in some way the entire file is a single PData or some attribute. Now the object gets loaded into memory (or many of these objects) get loaded into memory and the python process runs out of memory. So one of the quesitons: - What is approx. the largest instance in the ZODB? So I added min/max to the script. It may be useful to others (it is attached). What are some debugging tricks/reports you guys use to inspect the database? i.e. it would be really nice to be able to query the state of the ZODB, i.e. "show all records > 1MB". IIRC this should be doable in relstorage. But it would also be nice to have some set of tools for Filestorage as well. alan runyan analyze.py Description: Binary data ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] merging databases containing multidatabase references
> In ZODB 3.9, you can pack without doing garbage collection. This > should at least buy you time, if not solve your problem (depending on > how much garbage your application creates. What about zc.FileStorage? Does this support packing w/o GC? And does that work on versions of ZODB < 3.9? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] "Non-zeo version length" but ZMI says "There are no non-empty versions."
reported, https://bugs.launchpad.net/zodb/+bug/361184 maybe chris can take a stab at it *wink* On Tue, Apr 14, 2009 at 8:03 AM, Jim Fulton wrote: > > On Apr 13, 2009, at 9:34 PM, Chris Withers wrote: > >> Hi All, >> >> I get the following error when trying to open a filestorage .fs file >> with ZODB3-3.9.0a11: >> >> File "/ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/config.py", line >> 154, >> in open >> **options) >> File >> "/ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/FileStorage/ >> FileStorage.py", >> line 185, in __init__ >> read_only=read_only, >> File >> "/ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/FileStorage/ >> FileStorage.py", >> line 1560, in read_index >> h = fmt._read_data_header(pos) >> File >> "/ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/FileStorage/format.py", >> line >> 147, in _read_data_header >> h = DataHeaderFromString(s) >> File >> "/ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/FileStorage/format.py", >> line >> 231, in DataHeaderFromString >> return DataHeader(*struct.unpack(DATA_HDR, s)) >> File >> "/ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/FileStorage/format.py", >> line >> 241, in __init__ >> "Non-zero version length. Versions aren't supported.") >> ValueError: Non-zero version length. Versions aren't supported. >> >> However, if I open the same file in Zope 2.9.2 or Zope 2.11.0, the ZMI >> says theere are no non-empty versions. >> >> How do I get rid of this phantom version? > > > That's a great question. :) We probably need to write some sort of > utility to remove version records. > > Jim > > -- > Jim Fulton > Zope Corporation > > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - zodb-...@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] http://svn.zope.org/ZODB/trunk/src/ZEO/zrpc/smac.py
has an unused import of sha which is triggered loads of dep warnings ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB 3.9
> That would be great. If we're talking Python 2.6, it might be best to > wait until someone gets the tests passing first. Ok. I believe it would look like (only on Windows): Python 2.4, 2.5, 2.6 on 32bit Python 2.6 on 64bit FWIW: Python 2.6.1 < there is a .dll loading regression for Windows. Fixed in trunk. >> One problem is.. >> it takes ages for it the tests to run; > > About 45 minutes for me on a 32-bit VM with Python 2.5. Sounds right. 42 minutes on the 64bit VM. > You could email them to me, but please don't start emailing 2.6 tests > to me until we get them to pass. :) What about the zope-test mailing list? We are aiming to have this done by mid next week. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB 3.9
The tests were from ZODB trunk + Python 2.6 64bit on Windows. We could setup buildbot's for ZODB on Windows. One problem is.. it takes ages for it the tests to run; so it would need to be on its own VM. I think we could run the tests nightly.. where would you want the output? cheers alan p.s. no guarantees need to figure out how we could schedule them to run at night as to not interfere with our other builds On Tue, Apr 7, 2009 at 5:17 PM, Jim Fulton wrote: > > On Apr 7, 2009, at 6:05 PM, Alan Runyan wrote: > >>>> I was just reviewing what is pending for ZODB 3.9. >>>> https://bugs.launchpad.net/zodb/3.9/+bugs >>>> >>>> Looks like all of the bugs listed for 3.9 are fixed or fixes are >>>> known >>>> and pending. >>>> Is a ZODB 3.9 beta around the corner? >>>> >>>> According to changes.txt the first alpha of 3.9 was cut almost 1/2 >>>> year ago >>> >>> ZODB + Python 2.6 on Windows is appearently an open issue. >>> >>> Andreas >> >> Where are the buildbot's/test runners for ZODB? We just ran quick >> test and got the attached output. > > What platform and Python version was this for? > >> I presume ZC has buildbot's for >> ZODB already running? > > No. Do you? I typically run tests on Linux and Windows before > committing, usually for Python 2.4 and 2.5. Lately, also on Python 2.6 > on Linux. I would love someone to run some buildbots. I don't want to > set them up myself. :) > > Jim > > -- > Jim Fulton > Zope Corporation > > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - zodb-...@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB 3.9
>> I was just reviewing what is pending for ZODB 3.9.
>> https://bugs.launchpad.net/zodb/3.9/+bugs
>>
>> Looks like all of the bugs listed for 3.9 are fixed or fixes are known
>> and pending.
>> Is a ZODB 3.9 beta around the corner?
>>
>> According to changes.txt the first alpha of 3.9 was cut almost 1/2 year ago
>
> ZODB + Python 2.6 on Windows is appearently an open issue.
>
> Andreas
Where are the buildbot's/test runners for ZODB? We just ran quick
test and got the attached output. I presume ZC has buildbot's for
ZODB already running?
Tests with errors:
checkConcurrentUpdates2StoragesMT
(ZEO.tests.testConnection.FileStorageConnectionTests)
Tests with failures:
checkConcurrentUpdates2StoragesMT
(ZEO.tests.testConnection.FileStorageConnectionTests)
d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test
d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test
Total: 3594 tests, 3 failures, 1 errors in 42 minutes 23.752 seconds.
Failure in test checkConcurrentUpdates2StoragesMT
(ZEO.tests.testConnection.FileStorageConnectionTests)
Traceback (most recent call last):
File "c:\Python26\lib\unittest.py", line 279, in run
testMethod()
File
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line
425, in checkConcurrentUpdates2StoragesMT
self.go(stop, cd, t1, t2, t3)
File
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line
323, in go
t.cleanup()
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\TestThread.py", line
47, in run
self.testrun()
File
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line
54, in testrun
self._testrun()
File
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line
157, in _testrun
tree[key] = self.threadnum
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 808,
in setstate
self._setstate(obj)
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 855,
in _setstate
self._load_before_or_conflict(obj)
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 890,
in _load_before_or_conflict
if not self._setstate_noncurrent(obj):
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 913,
in _setstate_noncurrent
assert end is not None
AssertionError
(9.689 s)
Error in test checkConcurrentUpdates2StoragesMT
(ZEO.tests.testConnection.FileStorageConnectionTests)
Traceback (most recent call last):
File "c:\Python26\lib\unittest.py", line 289, in run
self.tearDown()
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\ConnectionTests.py",
line 163, in tearDown
os.unlink(path) # sometimes this is just gonna fail
WindowsError: [Error 32] The process cannot access the file because it is being
used by another process: '12-1.zec'
Failure in test
d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test
Traceback (most recent call last):
File "c:\Python26\lib\unittest.py", line 279, in run
testMethod()
File "c:\Python26\lib\doctest.py", line 2131, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for zeo_blob_cache.test
File
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test", line 0
--
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test",
line 72, in zeo_blob_cache.test
Failed example:
cache_size('blobs') < 5000
Expected:
True
Got:
False
Failure in test
d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test
Failed doctest test for zeopack.test
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test",
line 0
--
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test", line
140, in zeopack.test
Failed example:
main(["-ufoo", "-d3", "-t1:30", "host1:8100:2"])
Expected:
ClientStorage(('host1', 8100), read_only=1, storage='2', wait=False)
is_connected True
pack(1237624200.0, wait=True)
close()
ClientStorage('foo', read_only=1, storage='1', wait=False)
is_connected True
pack(1237624200.0, wait=True)
close()
Got:
ClientStorage(('host1', 8100), read_only=1, storage='2', wait=False)
is_connected True
pack(1237617000.0, wait=True)
close()
ClientStorage('foo', read_only=1, storage='1', wait=False)
is_connected True
pack(1237617000.0, wait=True)
close()
--
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test", line
158, in zeopack.test
Failed example:
main(["-d3", "-t1:30", "host1:8100:2"])
Expected:
ClientStorage(('host1', 8100), read_only=1, storage='2', wait=False)
is_connected False
sleep
[ZODB-Dev] ZODB 3.9
Hi. I was just reviewing what is pending for ZODB 3.9. https://bugs.launchpad.net/zodb/3.9/+bugs Looks like all of the bugs listed for 3.9 are fixed or fixes are known and pending. Is a ZODB 3.9 beta around the corner? According to changes.txt the first alpha of 3.9 was cut almost 1/2 year ago cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] create unique container keys
It seems like such an easy goal: autoincremental integers for a container. Is this such a problem because of the ZODB architecture? or lack there of? There are two database primitives that everyone appears to want: - autoincrementing integers for containers (tables) - indexes (not in application) The new generation of databases (couchdb, tokyo cabinet, hypertable, etc) autoincrementing is usually absent. Though these data containers usually have index support. cheers alan On Thu, Apr 2, 2009 at 7:48 AM, Stephan Richter wrote: > On Thursday 02 April 2009, Adam GROSZER wrote: >> Yeah, uuid was my guess. > > Christian was not necessarily thinking uuid. I think the common algorithm is: > > x = random.randint(10**9) > while str(x) in container: > x += 1 > > Regards, > Stephan > -- > Stephan Richter > Web Software Design, Development and Training > Google me. "Zope Stephan Richter" > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - zodb-...@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] upload a file of 6 MB with the function manage_upload
This is not a ZODB question. Ask on Zope User's mailing list. 2009/4/1 Sandra : > Hi all, > I want to upload a file of 6 MB with the function manage_upload > > > def > > manage_upload(self,file='',REQUEST=None): > > """ > > Replaces the current contents of the File or Image object with file. > > The file or images contents are replaced with the contents of 'file'. > > """ > > if self.wl_isLocked(): > > raise ResourceLockedError, "File is locked via WebDAV" > > data, size = self._read_data(file) > > content_type=self._get_content_type(file, data, self.__name__, > > 'application/octet-stream') > > self.update_data(data, content_type, size) > > if REQUEST: > > message= > > "Saved changes." > return self.manage_main(self,REQUEST,manage_tabs_message=message) > > in python/OFS/image.py. But my Programm run without end. > I'm making some mistake ? Anybody knows any way to solve > it ? > N.B: i am using Relstorage + Oracle 10i. > > Thanks in advance > > > _______ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - zodb-...@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Relstorage and ZEO
Ask this question on either Zope/Plone mailing list. My understanding about relstorage is that each Zope application client uses relstorage. There is no ZEO server in the mix. Is that correct Shane/others? cheers alan On Mon, Mar 30, 2009 at 9:36 AM, Anthony Gerrard wrote: > Hi, > > I'm working on a project to migrate a number of sites to a content management > system. We finished our evaluation period and chosen Zope / Plone. I'm new > to both products and would like some advice about our initial deployment. > The impression I get is that using both ZEO and Relstorage are recommended. > My question is whether you'd recommend using them together or not. > > The sites get up to half a million requests a day (not page impressions) and > we're planning on using Apache (mod_cache) and CacheFu for caching. > > Anthony > > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - zodb-...@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Relstorage pack problems
On Fri, Jan 23, 2009 at 1:45 PM, Shane Hathaway wrote: > Shane Hathaway wrote: > > Assuming your bad script caused your problem, it is likely that packing > > will still mess up your database, since you still probably have mixed-up > > object_state rows. Don't pack until I've had a chance to look again. > > Here is some more analysis. Now that I understand you accidentally > merged two databases into one by forcing copyTransactionsFrom() to run > when it shouldn't, I looked for the transactions you merged. First I > looked for the OIDs with a confused transaction ID. > > > => select zoid from current_object where tid != (select max(tid) from > object_state where object_state.zoid = current_object.zoid); > zoid > -- > 7 >10 >12 >11 > 9 > 8 > (6 rows) Wow. All of that with some SQL. Out of curiosity what would be the process if you were using FileStorage to diagnose the same problem? Do the same thing by iterating over the entire storage and filling in some structures and doing the comparison manually? Sounds like it would take a few hours, with SQL -- it seems pretty straightforward. I'm impressed. The sooner this gets into the mainline of ZODB -- the better. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
We run consistency checks on the ZODB - the same as we do on SVN -- nightly. http://svn.zope.org/ZODB/trunk/src/ZODB/scripts - fstest.py - checkbtrees.py - fsrefs.py We have never had corruption in years. But I just wanted to ask what people are doing to find any problems with consistency; so the problem can be solved ASAP. Miles, have you run any of these scripts across your production ZODB? cheers alan On Thu, Dec 4, 2008 at 12:45 PM, Dieter Maurer <[EMAIL PROTECTED]> wrote: > Miles wrote at 2008-12-4 13:39 +: >>I've moved a FileStorage from one (old) machine to another (new) >>machine, but when I mount it on the new machine I get a lot of time errrors: >> >>Traceback (innermost last): >> Module ZPublisher.Publish, line 115, in publish >> Module ZPublisher.mapply, line 88, in mapply >> Module ZPublisher.Publish, line 41, in call_object >> Module Shared.DC.Scripts.Bindings, line 311, in __call__ >> Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec >> Module App.special_dtml, line 176, in _exec >> Module DocumentTemplate.DT_Let, line 76, in render >> Module DocumentTemplate.DT_In, line 703, in renderwob >> Module DocumentTemplate.DT_With, line 76, in render >> Module DocumentTemplate.DT_Var, line 214, in render >> Module App.PersistentExtra, line 43, in bobobase_modification_time >> Module DateTime.DateTime, line 509, in __init__ >> Module DateTime.DateTime, line 760, in _parse_args >> Module DateTime.DateTime, line 437, in safelocaltime >>TimeError: The time 98040302366.810165 is beyond the range of this >>Python implementation. > > I expect that your storage is damanged and contains some garbage > at a place where a serial should be. > > This could probably be fixed -- but other parts of your storage > might be damanged, too. > > > > -- > Dieter > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Analyzing the committed data during one transaction
On Mon, Nov 17, 2008 at 9:01 AM, Jim Fulton <[EMAIL PROTECTED]> wrote: > > On Nov 17, 2008, at 8:21 AM, Andreas Jung wrote: > ... >> Rephrasing my question in a different way: how can I get hold of the >> "parent" object based a persistent object with a _p_jar >> connection? E.g. when the ZODB commits a subobject (e.g. a BTree or >> a bucket) of portal_catalog/Indexes/some_index then I would like to >> get hold of the the related parent "first-class" Zope 2 object (e.g. >> derived from SimpleItem/Item or something like that). > > There isn't a practical way to do that. (If you're willing to work > very very very hard, you can iterate over a database and build up a > reverese reference structure.) FWIW: I've wanted to do this same thing before but found no toolchain support and gave up. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] __setstate__ semantics
I would not imagine this functionality is not in scope of the ZODB. We were thinking about piggy backing on top of the invalidation messages sent from ZEO server at one point.. cant remember why we gave up on it. For various reasons: - The object may not be loaded on the "other" zeo client; therefore why should it be notified? - If you have a lot of changes it could generates oodles of messages; significantly degrading ZODB performance. The best thing is to look into a messaging library. AMQP is pretty rockin. I spoke w/ Stefan Richter and they are using Amazon's messaging platform. I believe what you are looking for is some sort of application level messaging across ZEO clients. Kapil has started an implementation at repoze.queue (I would prefer the name repoze.messaging) which may not yet have an implementation other than the interfaces and some tests. But this is a known problem (which is very hard to get right). cheers alan On Mon, Oct 20, 2008 at 2:36 PM, Roché Compaan <[EMAIL PROTECTED]> wrote: > If an object is modified on one ZEO client, will this always lead to a > __setstate__ call on this object on another ZEO client? > > If not, is there any event or method that I can hook into to determine > if an object was modifed by another ZEO client? > > -- > Roché Compaan > Upfront Systems http://www.upfrontsystems.co.za > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB Blob Question
Hi guys. At the plone sprint we started reviewing some code that uses Blob. The code that we reviewed is working in production although I wanted to be clear about the expected usage of Blob. ZODB/tests/blob_consume.txt says that you *must* close a file before calling consumeFile(). It seems the current Plone Blob implementation *does not close the file* before calling consumeFile. And all appears to be working. Why must the file be closed before calling consumeFile? And would you go as far as saying that if you *do not close* the file before calling consumeFile - all bets are off. You are not using Blob as intended? cheers, alan runyan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope memory usage
There was a recent modification to limit the ZODB cache to a set size. i.e. Limit the size of memory usage to 128MB. The original feature was implemented here: http://svn.zope.org/ZODB/branches/dm-memory_size_limited-cache/ You can get the feature+3.8 branch of the ZODB from: http://svn.zope.org/ZODB/branches/zcZODB-3.8/ The changes are also on trunk (will be ZODB 3.9). The goal of the modification is to try to put upper bounds on the size of the database cache. Before this "size limited cache" the cache would grow and if you had very large objects in the zodb cache -- your memory usage would be surpringsly high. Please use this feature in your testing at upfront. And let Roche know this feature has landed (if we was not monitoring svn checkins). The feature needs to be test driven. Give it a go and report your experience. cheers alan On Wed, Sep 17, 2008 at 5:10 AM, Izak Burger <[EMAIL PROTECTED]> wrote: > Hi all, > > I'm sure this question has been asked before, but it drives me nuts so I > figured I'll ask again. This is a problem that has been bugging me for > ages. Why does zope memory use never decrease? Okay, I've seen it > decrease maybe by a couple megabyte, but never by much. It seems the > general way to run zope is to put in some kind of monitoring, and > restart it when memory goes out of bounds. In general it always uses > more and more RAM until the host starts paging to disk. This sort of > baby-sitting just seems wrong to me. > > It doesn't seem to make any difference if you set the cache-size to a > smaller number of objects or use a different number of threads. Over > time things always go from good to bad and then on to worse. I have only > two theories: a memory leak, or an issue with garbage collection (python > side). > > It is possible that this is not a bug, my reasoning goes like this: The > OS exposes a "virtual memory" picture to the application, ie, the > application does not know how much of the RAM is real and how much is > paged out. All it does is call malloc every now and then. Combined with > a kernel that allows overcommit (which is in general a good thing), I > see this going only one way. > > I'm hoping someone on this list has some insights into the issue. > > regards, > Izak > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] wrap-up notes of sprint possible?
hey guys.. i know everyone is busy but it would be really swell if someone could write up a bullet list of things discussed/done at sprint. Things I'm curious about: - relstorage patch - zeoraid patch - transaction module (move to threading.local) https://bugs.launchpad.net/zodb/+bug/239086 - better import branch http://svn.zope.org/ZODB/branches/ctheune-betterimport/ cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Runaway cache size
I believe it is common to have a theshold (say every 1) objects to do a savepoint and then you can run cacheMinimize after the savepoint. surprised the zc.catalog does not provide this sort of knob. maybe it was a problem in the original ZCatalog implementation? cheers alan On Wed, Jul 30, 2008 at 1:21 PM, <[EMAIL PROTECTED]> wrote: > > I have a database consisting mainly of an IOBTree of ~700,000 items > (each persistent mappings), and an zc.catalog indexing them by one of the > mapping's fields (a date). I am opening the FileStorage in read_only mode. > For each day in the index, I get the day's mappings and read the contents > of another field. As I go through each day, the memory usage explodes (over > 32 GB). Is there a way to configure the cache to automatically keep itself > under the value of the cache size parameter? > > To avoid this problem in most cases, I wrap the IOBTree in another > object which does nothing more than call db.cacheMinimize after every 1 > items are iterated over. But for random access, that's not an option. > > From reading the archives, it sounds like cache cleaning does not > happen while running a transaction. Is that my problem? I'm in read-only > mode, so I can't perform a transaction anyway, but could the ZODB think I > want to? Or is my problem something else? > > Maybe if someone can point me to a description of how the caching > works. > > -- > Anthony Foglia > Princeton Consultants > (609) 987-8787 x233 > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
Chris, Are you seeing this error? https://bugs.launchpad.net/zodb/+bug/114814 If so maybe Jim's latest changes fixed this in ZODB3.8.1b65 -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB + ZEO performances problem
> - sometimes (but most often under quite heavy load) the Zope clients > completely hang and stop responding, without any access or error log, and > without any disk or CPU activity. I suppose it's a kind of deadlock, how can > I detect the source of the problem ? this isnt a zodb specific question. you will probably get more help moving this to the zope mailing list. check out DeadlockDebugger: http://highspeedrails.com/Support/Howto/deadlockDebuggerHowto cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] zeo clients seeing a different size of database?
Hi guys. This is more of a trivia question, I guess. Just saw something kinda strange. Customer had a zeo server where each 4 zeo clients connected. Some zeo clients were showing different sizes of the database. 5.2MB vs. 4.9MB. Not that this is a big deal at all. But it was kinda strange. It is possible for zeo clients to report different sizes of a ZEO server. Why? Running Zope 2.10.6; I believe that is the latest ZODB 3.7 series? -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Conflict Error
this sounds like a zope/plone problem not a ZODB problem. might want to ask on the relevant mailing list On Thu, Jun 5, 2008 at 5:10 AM, David Otero Figueroa <[EMAIL PROTECTED]> wrote: > > > Hello everyone, > > I have a couple questions related to "conflict errors". > > During our home page stress testing (20 concurrent users), we detected > several conflict errors (see trace below), We thought these could ONLY > appear when writing objects in the ZODB. > > 2008-05-27T18:57:53 INFO ZODB conflict error at /portal/ (26 conflicts since > startup at 2008-05-27T13:37:19) > > > I would like to know: > - Can conflict errors appear when reading objects from the ZODB? > - Could conflict errors be the cause of a "spinning ZOPE" scenario? > - Is there any situation where ZOPE could try to write objects when > rendering an "only read" page? > > Our configuration is as follows: Apache + Pound + ZEO architecture (5 ZEO > clients, 1 ZEO storage server - 650.000 ZODB objects) > > Thanks in advance. > > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Relstorage
This appears to be a CMFEditions problem not a RelStorage problem. The ways to escalate a bug: Write a test case that demonstrates the bug. Do it yourself. a. Ensure tests fail reliably b. Email the author with failing tests cases and software versions to allow him to reproduce c. Hope that arthur cares enough to run your tests on the software packages and fixes it. I have a feeling its CMFEditions because he is trying to do fancy stuff with the transaction note(s) in the ZODB. At a gut level: I feel 100x more comfortable with Shane's code (ZODB/RelStoage) than CMFEditions. Which is notoriously insane. alan On Wed, Apr 23, 2008 at 4:10 AM, Ben Mason <[EMAIL PROTECTED]> wrote: > > We're currently benchmarking RelStorage to see if it's a viable alternative > to ZEO. > > We've come across the following problem: When trying to revert to an old > version of a page, we get a unicode error: > > Traceback (innermost last): > Module ZPublisher.Publish, line 119, in publish > Module ZPublisher.mapply, line 88, in mapply > Module ZPublisher.Publish, line 42, in call_object > Module Products.CMFCore.FSPythonScript, line 140, in __call__ > Module Shared.DC.Scripts.Bindings, line 313, in __call__ > Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec > Module Products.CMFCore.FSPythonScript, line 196, in _exec > Module None, line 22, in revertversion > - > - Line 22 > Module Products.CMFEditions.CopyModifyMergeRepositoryTool, line 292, in > save > Module Products.CMFEditions.CopyModifyMergeRepositoryTool, line 454, in > _recursiveSave > Module Products.CMFEditions.ArchivistTool, line 288, in save > Module Products.CMFEditions.ZVCStorageTool, line 232, in save > Module Products.CMFEditions.ZVCStorageTool, line 391, in _applyOrCheckin > Module Products.CMFEditions.ZVCStorageTool, line 505, in _encodeMetadata > UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: > ordinal not in range(128) > > All that has been done here is... Click Edit on the front page of a clean > Plone site, click Save. Go to the History tab and click on 'Revert to this > revision' > > This doesn't happen if we're using the traditional Data.fs way of storing > data. > > Anyone know what is going on? > > Thanks > > Ben > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO Client deadlocking in asyncore.poll - how to I debug
check out zeo server log files. a known problem is people using iptables
or some sort of filtering between ZEO clients and ZEO server. this config
took several hours off my life ;-(
On Mon, Apr 7, 2008 at 9:16 AM, Anton Stonor <[EMAIL PROTECTED]> wrote:
> We have a setup with a ZEO server and 4 ZEO clients.
>
> During the last weeks we have seen almost daily deadlocks in some of the ZEO
> clients. I've tried to wait for up to 30 minutes before restarting a client.
>
> I could need an advice on how to debug this.
>
> With DeadlockDebugger I see the same pattern each time:
>
> One thread is hanging:
>
>
> File "/usr/local/www/zope-2.9.6/lib/python/ZODB/Connection.py", line 732,
> in setstate
>self._setstate(obj)
> File "/usr/local/www/zope-2.9.6/lib/python/ZODB/Connection.py", line 768,
> in _setstate
>p, serial = self._storage.load(obj._p_oid, self._version)
> File "/usr/local/www/zope-2.9.6/lib/python/ZEO/ClientStorage.py", line 746,
> in load
>return self.loadEx(oid, version)[:2]
> File "/usr/local/www/zope-2.9.6/lib/python/ZEO/ClientStorage.py", line 769,
> in loadEx
>data, tid, ver = self._server.loadEx(oid, version)
> File "/usr/local/www/zope-2.9.6/lib/python/ZEO/ServerStub.py", line 192, in
> loadEx
>return self.rpc.call("loadEx", oid, version)
> File "/usr/local/www/zope-2.9.6/lib/python/ZEO/zrpc/connection.py", line
> 531, in call
>r_flags, r_args = self.wait(msgid)
> File "/usr/local/www/zope-2.9.6/lib/python/ZEO/zrpc/connection.py", line
> 638, in wait
>asyncore.poll(delay, self._singleton)
> File "/usr/local/lib/python2.4/asyncore.py", line 122, in poll
>r, w, e = select.select(r, w, e, timeout)
>
>
> The other threads of the ZEO client are waiting for the hanging thread to
> release the storage lock so that they can acquire it:
>
> File "/usr/local/www/zope-2.9.6/lib/python/ZEO/ClientStorage.py", line 760,
> in loadEx
>self._load_lock.acquire()
>
>
> When I connect to the ZEO server monitor I can see an increasing number of
> reads (probably from the other ZEO Clients).
>
> I've set transaction-timeout 15.
>
> How to I dig further to resolve this?
>
> zeo.conf partly below:
>
> --
>
> address 8200
> read-only false
> invalidation-queue-size 100
> # pid-filename $INSTANCE/var/ZEO.pid
> monitor-address 8201
> transaction-timeout 15
>
>
>
> path $INSTANCE/var/Data.fs
>
>
> %import tempstorage
>
> name temporary storage for sessioning
>
> --
>
> Anton
>
>
>
> ___
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list - ZODB-Dev@zope.org
> http://mail.zope.org/mailman/listinfo/zodb-dev
>
--
Alan Runyan
Enfold Systems, Inc.
http://www.enfoldsystems.com/
phone: +1.713.942.2377x111
fax: +1.832.201.8856
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Hooking persistent.Persistent.__setstate__ was Re: [ZODB-Dev] Analyzing a ZODB.
Here is something more ZODB and less Zope related (kinda). I was talking with Benji a few weeks ago about a problem that should be easy to debug but was not. Here is the scenerio: - Customer has software on a remote machine. They are seeing unnecessary transaction commits. Just like the guy 'Analyzing a ZODB'. - Customer is completely incapable of doing anything other than putting a script on the filesystem. - Benji and I thought about it and he proposed 'the simplest thing that c/should work, monkey patch persistent.__setstate__' so that I could see what objects were being mutated. - Unfortunately ZODB 3.x does not have a Python fallback of persistent.Persistent -- its in C. The customer did not have a C compiler on their box. IIRC how I solved it was increase ZEO event log to see the oid's. Then I walked him through loading the oid up to see what object was being mutated. It was more painful than it should have been. Question: Is it possible for ZODB 3.9 to have a pure python implementation of persistent.Persistent? Maybe this would be a good ZODB GSOC project? Notes: - ZODB 4.x did have a pure python impl of persistent.Persistent but that project did not make it into production. - Seemingly Jim F. said there was a 'best practice' in writing C extensions in Python, first write reference impl in Python and to write tests around the impl. Then re-implement in C. ZODB 3.x, never had this 'best practice' put into play. - increasing zeo server log level and watching oid's being changed is sort-of the equivalent of turning on RDBMS logging to see SQL stmt's being executed. Unfortunately I believe without having a hook in persistent.Persistent we can never really get that level of granularity (i.e. __getattribute__ is only accessible in client) with only ZEO server logs. - We could "monkey patch" the subclasses that use persistent. But in the world of non-Zope2 applications usually there are very thin classes that may not have any common base class other than persistent.Persistent. This is a transparent plea for a "new feature". But I believe it would significantly help people writing ZODB applications. Maybe people should always have a base class that you override those methods which delegate to persistent.Persistent. i.e. class MyMixin(persistent.Persistent) and mixin MyMixin instead of mixin persistent.Persistent directly? Then you can instrument the MyMixin with the logging? Another GSOC idea: Best Practices of ZODB Programming. cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Hooking persistent.Persistent.__setstate__ was Re: [ZODB-Dev] Analyzing a ZODB.
Here is something more ZODB and less Zope related (kinda). I was talking with Benji a few weeks ago about a problem that should be easy to debug but was not. Here is the scenerio: - Customer has software on a remote machine. They are seeing unnecessary transaction commits. Just like the guy 'Analyzing a ZODB'. - Customer is completely incapable of doing anything other than putting a script on the filesystem. - Benji and I thought about it and he proposed 'the simplest thing that c/should work, monkey patch persistent.__setstate__' so that I could see what objects were being mutated. - Unfortunately ZODB 3.x does not have a Python fallback of persistent.Persistent -- its in C. The customer did not have a C compiler on their box. IIRC how I solved it was increase ZEO event log to see the oid's. Then I walked him through loading the oid up to see what object was being mutated. It was more painful than it should have been. Question: Is it possible for ZODB 3.9 to have a pure python implementation of persistent.Persistent? Maybe this would be a good ZODB GSOC project? Notes: - ZODB 4.x did have a pure python impl of persistent.Persistent but that project did not make it into production. - Seemingly Jim F. said there was a 'best practice' in writing C extensions in Python, first write reference impl in Python and to write tests around the impl. Then re-implement in C. ZODB 3.x, never had this 'best practice' put into play. - increasing zeo server log level and watching oid's being changed is sort-of the equivalent of turning on RDBMS logging to see SQL stmt's being executed. Unfortunately I believe without having a hook in persistent.Persistent we can never really get that level of granularity (i.e. __getattribute__ is only accessible in client) with only ZEO server logs. - We could "monkey patch" the subclasses that use persistent. But in the world of non-Zope2 applications usually there are very thin classes that may not have any common base class other than persistent.Persistent. This is a transparent plea for a "new feature". But I believe it would significantly help people writing ZODB applications. Maybe people should always have a base class that you override those methods which delegate to persistent.Persistent. i.e. class MyMixin(persistent.Persistent) and mixin MyMixin instead of mixin persistent.Persistent directly? Then you can instrument the MyMixin with the logging? Another GSOC idea: Best Practices of ZODB Programming. cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zeo Server as a single point of failure
> in that vein, DirectoryStorage-behind-ZEO has worked perfectly for > years... but this new stuff is very interesting! wishing we could get DirectoryStorage into svn.zope.org ~alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO+MultipleClients+ConflictErrors
Could you provide a bit more guidance? Maybe an example. Most people come at ZODB with previous experience in RDBMS. How do they map SQL INSERT/UPDATE activities to ZODB data structures? In a way that does not create hotspot. A few years ago people would say "pick a BTree implementation that maps to your data structures and use that." But that is naive and pushes the hotspot into the BTree where concurrent updates are happening. cheers alan On Tue, Feb 26, 2008 at 12:42 PM, Benji York <[EMAIL PROTECTED]> wrote: > Kenneth Miller wrote: > > Is there a standard bit of documentation as to how to handle a > > ZEO server with multiple clients writing to the same data structure at > > the same time? > > Not documentation, but there is a standard bit of snark that might be > helpful: "don't do that". ;) > > In other words, the best way to deal with a hot spot is to eliminate it. > That may or may not be possible, but it's a good thing to try. > -- > Benji York > Senior Software Engineer > Zope Corporation > > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO+MultipleClients+ConflictErrors
Kenneth, I believe you will want to look into 'conflict resolution'. Possible source of inspiration: http://www.zope.org/Documentation/Articles/ZODB2 -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] PGStorage
> Likely but I don#t know how to setup my instance in order to make the > copied instance making use of mounted catalog. What about creating a Plone site with a FileStorage. Then mount a subfolder into PGStorage say /Plone/some_folder then you could see create af older /Plone/new_folder and dump buncha content and paste it into /Plone/foo_folder and paste it into /Plone/some_folder and see the difference? > If there is a benefit..what would be arguments for running a mixed setup? My understanding is that each object in a catalog (Plone has 3 cataogs) has numerous other objects associated with it. Something like: each record in a catalog may have an object for each index/metadata attribute you are capturing. and possibly a few others. Each catalog entries contain ts of object per object being indexed.That is my understanding. -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] PGStorage
Andreas, Could you try to mount the catalog separately? My understanding is the catalog is what makes storages a misery. CMF/portal_catalog mounted as Filestorage and CMF could be mounted as PGStorage. I presume you would see much more reasonable performance? cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] PGStorage
Jim, What would you consider a "large active database"? curiously, alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] [FileStorage] "description too long" error
Leaky implementation details everywhere ;-(
Plone writes 'human legible' transaction notes. So the 'undo' screens in ZMI
were a bit more useful.
ZODB has a MAX transaction header length of 64k-1.
Plone's attempt to calculate the transaction header length seems to be
incorrect.
see CMFPlone.utils.transaction_note
ZODB's error handling of transaction header length may be incorrect as well:
the code in FileStorage.py:
def _begin(self, tid, u, d, e):
self._nextpos = 0
self._thl = TRANS_HDR_LEN + len(u) + len(d) + len(e)
if self._thl > 65535:
# one of u, d, or e may be > 65535
# We have to check lengths here because struct.pack
# doesn't raise an exception on overflow
if len(u) > 65535:
raise FileStorageError('user name too long')
if len(d) > 65535:
raise FileStorageError('description too long')
if len(e) > 65535:
raise FileStorageError('too much extension data')
1. What if each u, d, e were each 32k? this exception would
not be triggered.
2. Should there be a more robust calculation in
ZODB.FileStorage.FileStorage_begin to guarantee an exception?
3. There certainly should be a raise FileSorageError('transaction
header too long')
if none of the if len(e) > 64k-1 are triggered.
Andreas,
CMFPlone.utils.transaction_note
if (len(T.description)+len(note))>=65535:
log('Transaction note too large omitting %s' % str(note))
Note sure how description got to be > 64k-1. since in Plone
it does not write the transaction note under such circumstance.
cheers
alan
-- Forwarded message --
From: Andreas Jung <[EMAIL PROTECTED] >
Date: Jan 12, 2008 8:09 AM
Subject: [ZODB-Dev] [FileStorage] "description too long" error
To: zodb-dev@zope.org
Hi,
running Zope 2.10.5 with Plone 3.0.5. While performing a workflow transition
for all subjects within a tree (roughly about 3000 objects) I encountered
the following error:
:8080/20080112-123756/content_status_history
Traceback (innermost last):
Module ZPublisher.Publish, line 125, in publish
Module Zope2.App.startup, line 238, in commit
Module transaction._manager, line 96, in commit
Module transaction._transaction, line 395, in commit
Module transaction._transaction, line 493, in _commitResources
Module ZODB.Connection, line 494, in tpc_begin
Module ZODB.BaseStorage, line 238, in tpc_begin
Module ZODB.FileStorage.FileStorage, line 863, in _begin
FileStorageError: description too long
What's going on here?
Andreas
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
--
Alan Runyan
Enfold Systems, Inc.
http://www.enfoldsystems.com/
phone: +1.713.942.2377x111
fax: +1.832.201.8856
pgpkLIYTK5jGg.pgp
Description: PGP signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zeopack requirements?
Makes sense. Is zc.FileStorage an alternative packing strategy? Does it use less memory? http://svn.zope.org/zc.FileStorage/dev/src/zc/FileStorage/ cheers alan On Dec 17, 2007 8:29 AM, Jim Fulton <[EMAIL PROTECTED]> wrote: > > On Dec 17, 2007, at 9:14 AM, Alan Runyan wrote: > ... > > not believe zeopack will bring the entire database into memory at > > once. It may/ > > probably brings the index into memory. > > > > Suggestion: run zeopack and zeoserver on a separate machine or > > allocate more > > memory to your zeoserver process. > > > zeopack doesn't consume any reasources to speak of. It just tells the > *server* to pack. The resources are consumed by the server. > > > Jim > > -- > Jim Fulton > Zope Corporation > > > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zeopack requirements?
You are running zeopack on the production machine with "many clients"? Are you running the zeoserver on the same machine, "with many clients"? It seems it is the machine that ZEO server is on that is throwing the MemoryError. I do not believe zeopack will bring the entire database into memory at once. It may/ probably brings the index into memory. Suggestion: run zeopack and zeoserver on a separate machine or allocate more memory to your zeoserver process. Zope NOTE: Running zeo clients with zserver-threads 2 is common and rarely does this impact performance. you will consume much less memory per zeo client than the 'out of the box' zope.conf which specifies zserver-threads 4 (I am -1 on this default) cheers alan On Dec 17, 2007 6:36 AM, Chris Bainbridge <[EMAIL PROTECTED]> wrote: > I have a ~9GB live ZEO database. Last night I tried to pack it, but > got this error: > > 2007-12-17T01:27:08 (4179) Error raised in delayed method > Traceback (most recent call last): > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZEO/StorageServer.py", > line 1016, in run > result = self._method(*self._args) > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZEO/StorageServer.py", > line 351, in _pack_impl > self.storage.pack(time, referencesf) > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZODB/FileStorage/FileStorage.py", > line 1352, in pack > opos = p.pack() > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZODB/FileStorage/fspack.py", > line 482, in pack > self.gc.findReachable() > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZODB/FileStorage/fspack.py", > line 228, in findReachable > self.findReachableAtPacktime([z64]) > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZODB/FileStorage/fspack.py", > line 315, in findReachableAtPacktime > self.reachable[oid] = pos > File > "/home/user/lib/python/ZODB3-3.7.2-py2.4-linux-i686.egg/ZODB/fsIndex.py", > line 84, in __setitem__ > tree[key[6:]] = value > MemoryError > 2007-12-17T01:27:16 (7412) opening storage '1' using FileStorage > 2007-12-17T01:34:25 datastore.fs truncated, possibly due to damaged > records at 8580513351 > 2007-12-17T01:34:25 Writing truncated data from datastore.fs to > datastore.fs.tr0 > > Are there any limits on the pack operation? I assume that it doesn't > have to hold the entire DB in memory at once? Would bringing the > database offline and then packing it work (ie. we run out of memory > because we have many active clients), or is there some fundamental > constraint in the underlying code? > > Thanks for your help, > Chris > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Updating ZEO cache file magic number to get rid of version info
+1 On Nov 9, 2007 9:46 AM, Jim Fulton <[EMAIL PROTECTED]> wrote: > > I'm studying the ZEO cache-file implementation. I'd like to remove > version support from the trunk. Would anyone object if I changed the > magic number on the trunk (3.9) to reflect that cache data records no- > longer contain version information? This would mean that persistent > cache files created with 3.8 and earlier wouldn't work with 3.9. > > Jim > > -- > Jim Fulton > Zope Corporation > > > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Recovering from BTree corruption
> And, as you said in another node, the BTree folder actually loves in
> the resources database.
Correct the BTree is in /plone/resources/files to be exact.
> Cross database references are inherently weak. A reference from a
> foreign database doesn't prevent an object from being treated as
> garbage. So, if the only reference to an object is from a foreign
> database, then the object is considered garbage. It doesn't sound
> like this is what's affecting you. The cross-database reference is
> to the BTree. It sounds like the internal references are within
> database.
Well. Someone could have 'copy/pasted' a file from the content database
into the resources/files database. That could have been one issue.
> > - checkbtrees.py
> > - fstest.py
>
> There's an fsrefs script that checks internal references I believe.
fsrefs.py shows loads of problems in both the data.fs and the resources.fs.
probably > 200 entries per database. i.e.
oid 0xD87110L BTrees._OOBTree.OOBucket
last updated: 2007-09-04 14:43:37.687332, tid=0x37020D3A0CC9DCCL
refers to invalid objects:
oid ('\x00\x00\x00\x00\x00\xb0+f', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xb0N\xbc', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xb0N\xbd', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xd7\xb1\xa0', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xc5\xe8:', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xc3\xc6l', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xc3\xc6m', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xcahC', None) missing: ''
oid ('\x00\x00\x00\x00\x00\xaf\x07\xc1', None) missing: ''
My questions are:
- I imagine if there are 'invalid' references this is considered "corruption"
or "inconsistency"?
- How do I tell if something is a reference to another database?
- Having these invalid references, is this common to ZODB applications?
> Possibly, there's a backup that has data records for the missing OIDs.
Going to ask hosting company to pull up backups for the past few weeks.
But how i'm going to find this other than "seeing if the folder allows me
to iterate over the items" is not throwing POSKeyError. Does that sound
like a decent litmus test?
--
Alan Runyan
Enfold Systems, Inc.
http://www.enfoldsystems.com/
phone: +1.713.942.2377x111
fax: +1.832.201.8856
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Recovering from BTree corruption
Hi guys. It seems that one of our customers has a corrupted BTree. I would love for someone to provide some insight on how we can recover the data. we have two databases: 1 for resources and 1 for 'content'. resources contain lots of very big files. The system is configured to have a mount point at /plone/resources is a subclass of BTreeFolder, using internal data struct of OOBTree. anytime I iterate over the keys I get POSKeyError. anytime I iterate over the values the same. if I run BTree.check() on the data structure's tree attribute (the OOBTree itself) I get a POSKeyError. Running the utils.checkbtrees doesnt say this btree has a problem. While debugging this I had a conversation with sidnei about mounted databases. He recalled that if your using a mounted database you should not pack. If for some reason your mounted database had a cross reference to another database and somehow you had a dangling reference to the other database it would cause POSKeyError. Is there any other ways of "testing consistency" of FileStorage other than: - checkbtrees.py - fstest.py And any ideas how I can salvage the data? This BTree, of course, had the most valuable data. cheers -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Memory Errors reading large ZODB
snip... > line 96, in > __setstate__ > Persistent.__setstate__(self, state) > MemoryError means you have run out of memory. check your jail or how much memory you have allocated. you can call cache minimize after a threshold.. maybe every 100 iterations. there are finer grain mechanisms to do this but i cant remember them off the top of my head. http://svn.zope.org/ZODB/trunk/src/ZODB/interfaces.py cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] exporting zope data as sql
This is not the appropriate forum; please use zope-users mailing list. On 5/17/07, Freezin Monkie <[EMAIL PROTECTED]> wrote: Hi all, I just joined the mailing list. I'm trying to export data stored in zope as sql. Is this possible and what's the best way to do this. Also is there a way to restrict which data is exported, i'm interested in exporting the data store through a particular zope product only. _ The next generation of Hotmail is here! http://www.newhotmail.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] zeo bug
Just got this on a website, Zope 2.9.6. Looks like /tmp is 90% full, not sure that would cause the problem? Not using persistent disk cache: mount-point / # ZODB cache, in number of objects cache-size 2 server $HOST:$PORT storage 1 name $MAIN_NAME var $INSTANCE/var # ZEO client cache, in bytes cache-size 1024MB # Uncomment to have a persistent disk cache #client zeo1 mount-point /audioholics/portal_catalog container-class Products.CMFPlone.CatalogTool.CatalogTool cache-size 5 cache-size 1024MB server $HOST:$PORT storage 2 name $CATALOG_NAME var $INSTANCE/var KeyError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 196, in publish_module_standard * Module Products.PlacelessTranslationService.PatchStringIO, line 34, in new_publish * Module ZPublisher.Publish, line 146, in publish * Module Zope2.App.startup, line 222, in zpublisher_exception_hook * Module ZPublisher.Publish, line 106, in publish * Module ZPublisher.BaseRequest, line 366, in traverse * Module ZODB.Connection, line 732, in setstate * Module ZODB.Connection, line 786, in _setstate * Module ZODB.serialize, line 604, in setGhostState * Module ZODB.serialize, line 597, in getState * Module ZODB.serialize, line 471, in _persistent_load * Module ZODB.serialize, line 537, in load_oid * Module ZODB.Connection, line 201, in get * Module ZEO.ClientStorage, line 746, in load * Module ZEO.ClientStorage, line 774, in loadEx * Module ZEO.cache, line 293, in store * Module ZEO.cache, line 980, in add * Module ZEO.cache, line 915, in _makeroom KeyError: 131867244 (Also, the following error occurred while attempting to render the standard error message, please see the event log for full details: already have current data for oid) -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Cleaning up storage interfaces
I just checked out DirectoryStorage and grepped for getSerial - no occurances. +1 for cleaning up the storage interfaces.. was looking at it over xmas holidays to see if I could implement a sqlite version that was undoless. I didnt get far ;( alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] another reason to stop supporting versions
+1 for removing versions. they have been considered "bad practice" for over 3 years. I believe 100% its ok to remove them. alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RESOLUTION: Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
Thanks to Jim, Theune, Dieter and all others who weighed in on this thread. The problem: ZEO Clients would lock up randomly requiring restart. The hints: Lots of 'Connection timed out' and 'No route to host' in ZEO Server log files. The solution: The machine the ZEO server was running iptables. Even though it was allowing the zeo server port 8100; it was filtering on all interfaces. I simply changed rules so port 8100 would not filter the internal network interface - at all. ZEO listens specifically on internal ip/port 8100. System has been stable for several days; without zeo server logs containing any connection errors. I had never had this problem before because.. well.. we use firewalls on our customers (external interfaces and our internal interfaces never have filtering) and this particular sysadmin was running iptables on all public/private interfaces; locking the machine down as much as possible. Thanks again guys! -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
> Storage: 1 > Server started: Wed Apr 11 10:56:50 2007 > Clients: 10 > Clients verifying: 0 > Active transactions: -1 Huh? You're owing the system a transaction. However, by looking at the code briefly, this might happen if tpc_abort() and _abort() kind of overlap. And you did have two aborts at that point in time. Sounds like a bug/race that needs to be looked into in ZEO. Ah. The different clients might be because you have two storages and your ZEO clients are configured in a way not to connect to the exactly same storages? Or they are but they weren't able to. (See hardware/network problems.) They are both defined in zope.conf. All 12 clients were restarted last night: Just now I'm seeing: Storage: 1 Server started: Wed Apr 11 10:56:50 2007 Clients: 12 Clients verifying: 0 Active transactions: -1 Commits: 92 Aborts: 2 Loads: 498120 Stores: 2279 Conflicts: 0 Conflicts resolved: 20 Storage: 2 Server started: Wed Apr 11 10:56:50 2007 Clients: 11 Clients verifying: 0 Active transactions: 0 Commits: 51 Aborts: 0 Loads: 225080 Stores: 6408 Conflicts: 0 Conflicts resolved: 167 Something that came to my mind that might block the ZEO server for a long time are hard disk failures. Check your dmesg log. However, the network errors you see in various places really need to be tracked down. nothing in dmesg. I find the 'No route to host' disturbing although these have not happened over the past 24 hours. This has: 2007-04-12T00:17:45 ERROR ZEO.zrpc.Connection(S) (172.16.235.120:54881) Error caught in asyncore Traceback (most recent call last): File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 69, in read obj.handle_read_event() File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 391, in handle_read_event self.handle_read() File "/usr/local/zope/lib/python/ZEO/zrpc/smac.py", line 147, in handle_read d = self.recv(8192) File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 343, in recv data = self.socket.recv(buffer_size) error: (110, 'Connection timed out') -- which is frustrating. as i understand zeoserver is taking too long to communicate to zeoclient and zeoclient times out. shouldnt it retry / reconnect? alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
ETHERNET CARD: 05:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 11) - I have turned down transaction timeout to 15 seconds. - I currently have 11 ZEO Clients up; but showing below (seems strange) sometimes the clients are 6 or 7.. - Have not confirmed ZEO MONITOR OUTPUT: Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. ZEO monitor server version 3.6.2 Wed Apr 11 12:30:31 2007 Storage: 1 Server started: Wed Apr 11 10:56:50 2007 Clients: 10 Clients verifying: 0 Active transactions: -1 Commits: 24 Aborts: 2 Loads: 95941 Stores: 523 Conflicts: 0 Conflicts resolved: 20 Storage: 2 Server started: Wed Apr 11 10:56:50 2007 Clients: 8 Clients verifying: 0 Active transactions: 0 Commits: 9 Aborts: 0 Loads: 40836 Stores: 714 Conflicts: 0 Conflicts resolved: 167 -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
I'ev not had anything but bad experiences with pound myself, lvs seems a much more preferable alternative... We have not had such negative experiences with pound. >data = self.socket.recv(buffer_size) > error: (113, 'No route to host') That *is* very odd, anything other than pound being used for load balancing or traffic shaping? This has to be a major problem maker in the system. Pound is simply round robin connections to pool of Zope. > File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 343, in rec > v >data = self.socket.recv(buffer_size) > error: (110, 'Connection timed out') Storage server too busy :-( the storage server can be "too busy" when its only reading? cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
How does data get into the ZEO storage then? We have 10 ZEO clients that are for public consumption "READ ONLY". We have a separate ZEO client that is writing that is on a separate box. The website got slashdoted this morning and we had 4 zeo clients go out. Basically waiting for the zeo server for many minutes (basically hung and reporting 500s back to browsers). 4/10/07 - delta ah2 lock up - http://paste.plone.org/13919 4/10/07 - epsilon ah2 lock up - http://paste.plone.org/13920 4/10/07 - delta ah1 lock up - http://paste.plone.org/13936 4/10/07 - epsilon ah3 lock up - http://paste.plone.org/13935 Just FYI: Varnish didnt go over 3% CPU during the traffic surge; over 200 req/second. The ZEO Server was consistently at 2% CPU. Lots of traffic was getting through the cache and back to pound which was then load balancing to the ZEO clients. For Jim: We did not adjust the transaction timeout. Would that have helped in the case of READ's? The customer was posting content throughout the slashdot. The problem was that when the clients would update they would end up through 500s I am seeing something *very* strange in zeo.log: 2007-04-10T12:20:53 ERROR ZEO.zrpc.Connection(S) (172.16.235.120:49351) Erro r caught in asyncore Traceback (most recent call last): File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 69, in read obj.handle_read_event() File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 391, in han dle_read_event self.handle_read() File "/usr/local/zope/lib/python/ZEO/zrpc/smac.py", line 147, in handle_re ad d = self.recv(8192) File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 343, in rec v data = self.socket.recv(buffer_size) error: (113, 'No route to host') but several more of these: 2007-04-10T13:55:36 ERROR ZEO.zrpc.Connection(S) (172.16.235.119:44322) Erro r caught in asyncore Traceback (most recent call last): File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 69, in read obj.handle_read_event() File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 391, in han dle_read_event self.handle_read() File "/usr/local/zope/lib/python/ZEO/zrpc/smac.py", line 147, in handle_re ad d = self.recv(8192) File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 343, in rec v data = self.socket.recv(buffer_size) error: (110, 'Connection timed out') -- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
> Try setting the transaction timeout. Say, maybe, 10 seconds. Do you have anything that is committing very large transactions? No. In fact; these clients could be running in read only mode. As far as I'm concerned. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
Hi Jim, yes, still having lockups. We have a list of deadlockdebugger info associated with each lockup. The client configuration as at the beginning of this thread. The zeo server configuration is as follows: # ZEO configuration file %define INSTANCE /usr/local/zeo-ah address 8100 read-only false invalidation-queue-size 100 # pid-filename $INSTANCE/var/ZEO.pid # monitor-address PORT # transaction-timeout SECONDS path $INSTANCE/var/Data.fs path $INSTANCE/var/Catalog.fs level info path $INSTANCE/log/zeo.log program $INSTANCE/bin/runzeo socket-name $INSTANCE/etc/zeo.zdsock daemon true forever false backoff-limit 10 exit-codes 0, 2 directory $INSTANCE default-to-interactive true # user zope python /usr/local/python/bin/python zdrun /usr/local/zope-2.9.6/lib/python/zdaemon/zdrun.py # This logfile should match the one in the zeo.conf file. # It is used by zdctl's logtail command, zdrun/zdctl doesn't write it. logfile $INSTANCE/log/zeo.log any insight would be appreciated. cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2
Hi guys. Running Zope 2.9.6 with ZODB 3.6.2 on Python 2.4 Having lots of lockups. Have approximately 12 zeo clients on 2 machines connecting to a single zeo server. All on local network. Disks and network is monitored by competent hosting company. All looks healthy except zeo communication. Linux 2.6.9-42.0.8.ELsmp some feedback from manage_debug_threads: http://paste.plone.org/13821 http://paste.plone.org/13822 http://paste.plone.org/13823 typical client zeo configuration: /var/zope/ah0/zope.conf (included zope.conf) %define HTTP_SERVER 7580 %define ICP_SERVER 7680 %define INSTANCE /var/zope/ah0 %define MAIN_NAME epsilon0 %define CATALOG_NAME epsilonA %include /var/zope/etc/zope.conf /var/zope/etc/zope.conf (main zope.conf) mount-point / # ZODB cache, in number of objects cache-size 2 server gamma-gw.audioholics.com:8100 storage 1 name $MAIN_NAME var $INSTANCE/var # ZEO client cache, in bytes cache-size 1024MB # Uncomment to have a persistent disk cache #client zeo1 mount-point /audioholics/portal_catalog container-class Products.CMFPlone.CatalogTool.CatalogTool cache-size 5 cache-size 1024MB server gamma-gw.audioholics.com:8100 storage 2 name $CATALOG_NAME var $INSTANCE/var any ideas? alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] zeo cache hang?
Hi guys. Got this on, http://paste.plone.org/13772 The entire zope comes to a halt. any ideas? cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
