Re: [ZODB-Dev] Making ZODB / ZEO faster
On Fri, Dec 4, 2009 at 9:41 AM, Erik Dahl ed...@zenoss.com wrote: I'm guessing the indexes are a hotspot (haven't tested this out much though I guess b-tree's buckets should alleviate this problem some). (is there a persistent queue around?) Cataloging certainly can be a hot spot. Check out zc.catalogqueue. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Python 3
On Fri, Dec 5, 2008 at 11:15 AM, Adam [EMAIL PROTECTED] wrote: Is there a roadmap for getting ZODB to work on Python 3 or is that beyond the realms of possibility? Possibly, using a Python 2 ZEO server and connecting using a Python 3 ClientStorage? I can't speak for Jim, but here's my thoughts, anyway. The biggest hurdles for packages that have substantial test suites and development tool support (like ZODB) will be the availability of tools. At this point, setuptools and zc.buildout don't support Python 3 yet; until setuptools is available there's unlikely to be any movement to supporting Python 3 in the Zope world. Hopefully, someone will deal with that fairly soon, but I can't bring myself to hold my breath for it. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Thu, Jul 31, 2008 at 11:37 AM, Chris Withers [EMAIL PROTECTED] wrote: However, I suspect the difference-that-makes-a-difference is the persistent cache files. With non-persistent cache files, iirc, the files are stored in /tmp as temp files under unique names, so this kind of problem can't happen. As Jens noted, this is because you're sharing a cache file. That's one of those don't do that things. It's often useful to have a separate instance that's used for run scripts and debugging; configuring that not to use a persistent cache is easy, and essential if you want to be able to run any number of them at a time. -Fred -- Fred L. Drake, Jr. fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Thu, Jul 31, 2008 at 2:15 PM, Andreas Jung [EMAIL PROTECTED] wrote: They aren't part of the ZODB 3.8 but part of the trunk/3.9 - right? These are bug fixes, and will be included in ZODB 3.8.1. -Fred -- Fred L. Drake, Jr. fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] zc.recipe.zeo
On 8/3/07, Jens Vagelpohl [EMAIL PROTECTED] wrote: Just setting the number of threads to 1 won't improve anything. Their strategy is to use many instances with a single thread instead of a few instances with the default number of threads. For a single instance you'll do more harm than good. Yes, that's an important point. If there's only one instance, you'll need many threads, or you could see problems /very/ quickly. I'd been assuming David was interested in large-volume and/or high availability installations, given that he mentioned attending the high-performance session at EuroPython. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: [Zope-Annce] Technical Preview of Blob support in ZODB
On Thursday 18 May 2006 14:34, Fred Drake wrote: Have there been any new features since 3.6? I wonder if maintaining yet another branch can be avoided. On 5/20/06, Stephan Richter [EMAIL PROTECTED] wrote: I dunno; I just follow the pattern. ;-) Ok, I took a (very quick) look at the differences between the 3.6 and 3.7 branches; there are definately differences. I don't pretend to really understand these changes: - addAfterCommitHook added to ITransaction - record_iternext was added to the storage inteface (just for ZEO?) So we do need a 3.7 branch after all. :-( -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Education is hanging around until you've caught on. -- Robert Frost ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: [Zope-Annce] Technical Preview of Blob support in ZODB
On 5/18/06, Stephan Richter [EMAIL PROTECTED] wrote: I have already created the 3.7 branch, so no more new features in 3.7. However, ZODB 3.7 might be an internal release only used by Zope 3.3 and 2.10. Have there been any new features since 3.6? I wonder if maintaining yet another branch can be avoided. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Education is hanging around until you've caught on. -- Robert Frost ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: 64-bit BTrees
On 4/13/06, Fred Drake [EMAIL PROTECTED] wrote: I've created a feature development branch for this, and checked in my initial implementation. I've made another branch for this, with a different twist. I'm not sure it'll be interesting, but I think it'll solve my immediate need until I can get around to reasonable testing of the performance implications. The fdrake-optional-64bits branch will compile using the C int type for I keys and values by default, and using the PY_LONG_LONG type if ZODB_64BIT_INTS is defined. This allows 64-bit BTrees by building ZODB like this: python setup.py build_ext -D ZODB_64BIT_INTS build BTrees.IIBTree.using64bits will be True if ZODB_64BIT_INTS is used. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Don't let schooling interfere with your education. -- Mark Twain ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: [Zope-dev] Re: 64-bit BTrees
On 4/17/06, Jim Fulton [EMAIL PROTECTED] wrote: The fact that IIBTrees is so widely used is exatly the reason I want to use 64-bits for the existing types rather than having to introduce a new type. Oops, I was checking in the separated version of 64-bit BTrees while this was landing in my inbox. ;-/ Once we determine which approach we're going with, I should make an alpha release of ZODB 3.7 and knit that into the Zope 3 trunk so we can get more testing in context. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Don't let schooling interfere with your education. -- Mark Twain ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] 64-bit BTrees
I have a need for 64-bit BTrees (at least for IOBTree and OIBTree), and I'm not the first. I've created a feature development branch for this, and checked in my initial implementation. I've modified the existing code to use PY_LONG_LONG instead of int for the key and/or value type; there's no longer a 32-bit version in the modified code. Any Python int or long that can fit in 64 bits is accepted; ValueError is raised for values that require 65 bits (or more). Keys and values that can be reported as Python ints are, and longs are only returned when the value cannot be converted to a Python int. This can have a substantial effect on memory consumption, since keys and/or values now take twice the space. There may be performance issues as well, but those have not been tested. There are new unit tests, but more are likely needed. If you're interested in getting the code from Subversion, it's available at: svn://svn.zope.org/repos/main/ZODB/branches/fdrake-64bits/ Ideally, this or some variation on this could be folded back into the main development for ZODB. If this is objectionable, making 64-bit btrees available would require introducing new versions of the btrees (possibly named LLBTree, LOBTree, and OLBTree). I welcome comments. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Don't let schooling interfere with your education. -- Mark Twain ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
