[ZODB-Dev] RE: RE: PersistentMapping
Tim Peters wrote: I'm the closest thing to a ZODB maintainer there is, and I won't object ;-) Go for it! OK. Right now I have the problem of getting the tests to pass before I start changing things. Doing just as README.txt says (python2.4 setup.py build, then python2.4 test.py) earns me waggon loads of errors which almost entirely seem to stem from the fact that the transaction module is not available. Is there a reason the module is not in build/lib.foo after running the setup.py build? If you count this as a new feature, it should only go in on the ZODB trunk. If you count this is a bugfix (that's my inclination), And mine. then it should also go in: ZODB/branches/3.4 ZODB/branches/3.5 ZODB/branches/3.6 An almost disencouraging number of branches ;o) Since the database root object under all ZODBs is of type PersistentMapping, and everyone knows that, at this point I'd prefer to deprecate PersistentDict (under the sensible theory that there was never a real reason for it to exist in the ZODB 3 line). +1 -- Thomas ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Migration of obsolete BTrees to ZOPE 2.8
[Margie Watkins CONT] The issue http://www.zope.org/Collectors/Zope/1695 asks for a statement from users who are affected by the missing modules. Unfortunately, that's become out of sight, out of mind since then -- the issue was opened 9 months ago, and to date you're the first to say you really need this (if you read through all the referenced threads, you'll see that everyone else who thought they _might_ need it eventually discovered that they did not). It is a genuine problem for us. We have a large user base which has instances of obsolete BTrees and intSets. We need to be able to read those objects and migrate them to a new release of our product, WISE. I am unable to migrate them because of the different state formats in BTree. The convert method mentioned in the referenced thread does not help. I'm not sure to what that refers, but not sure it matters either. There are four threads referenced in the bug report, and AFAICT none of them claim to present a migration method for old BTrees or intSets. Jim Fulton's message here is the clearest on that topic: http://mail.zope.org/pipermail/zodb-dev/2005-February/008496.html ... it won't help with the uses of the ancient obsolete BTrees. They aren't the same as the new BTrees but in a different location. They are actually totally differeent types with different state formats. The new BTree classes cannot deal with their state. ... I fear we'll have to write some sort of conversion script to deal with converting old-style BTrees to new-style BTrees. Another idea might be to define some sort of protocol for dealing with unrecognized classes. When the database encounters a class it doesn't recognize, it could look in a table for a handler. That handler would be responsible for creating a new object. Perhaps this could be combined with some sort of database conversion process. Since Zope Corp didn't have this need in its own deployments, and no users spoke up (at the time, or in the nine months after) with this need either, no work was done by Zope Corp on this, and no work is currently planned. It's possible that _someone_ wrote a conversion script, but if so I haven't heard about it. I also need to be able to read the intSet. Right, they have the same kinds of issues as the old BTrees (totally different type with different state format), and so there's no dead easy way to convert them either. Bottom line is that you need conversion code that (probably) doesn't exist. Anyone here want to write it? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RE: RE: PersistentMapping
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tim Peters wrote: So I tried that just now using ZODB trunk. It failed too, but with different symptoms than you're describing: $ python2.4 setup.py build ... $ python2.4 test.py -v Running tests from build/lib.linux-i686-2.4 Traceback (most recent call last): File test.py, line 40, in ? from zope.testing import testrunner ImportError: No module named zope.testing $ I never do setup.py build myself, so I was only mildly surprised by that. It should probably be fixed ;-) Note I added couple of weeks ago a comment about this within the README.txt of ZODB. To test the build, first build the externals and add the src dir to your PYTHONPATH, % python setup.py build_ext -i % export PYTHONPATH=`pwd`/src:$PYTHONPATH Then run the test script:: % python test.py Cheers, J. - -- Julien Anguenot | Nuxeo RD (Paris, France) CPS Platform : http://www.cps-project.org Zope3 / ECM : http://www.z3lab.org mail: anguenot at nuxeo.com; tel: +33 (0) 6 72 57 57 66 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDffbbGhoG8MxZ/pIRAmLCAJ9Pjq4FqgbHIOI11V5zyfjONf3xMACfbEmO RV9RuLydwQvLEz/VAUZT2k4= =vNBY -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RE: RE: RE: PersistentMapping
Tim Peters wrote: Things I can't guess include which version of ZODB you're trying this with, and exactly what the errors were. Copy+paste generally works a lot better than English paraphrasing. Sorry. From the build/lib.foo part I guess you're running on Linux. Right. So I tried that just now using ZODB trunk. It failed too, but with different symptoms than you're describing: $ python2.4 setup.py build ... $ python2.4 test.py -v Running tests from build/lib.linux-i686-2.4 Traceback (most recent call last): File test.py, line 40, in ? from zope.testing import testrunner ImportError: No module named zope.testing $ Well, as I wrote, I have the zope package from a current Zope3 trunk in my path. Oh, wait. I wrote this on #zope3-dev where I asked first. Sorry again, I confused myself by spreading over too many (i.e. more than one) media... Anyway, you don't seem to have Zope in your path, but I do. This worked (which is what I normally do): It did for me too. I guess the crucial difference was doing build_ext instead of just build. Maybe someone with more distutils clue could say something about it; if my guess is right, README.txt should be changed. It should also go in the Zope-2_7-branch branch of CVS module ZODB3 (which will eventually become ZODB 3.2.11, for use in the Zope 2.7 line). Yes, you wrote that before, I just didn't quote it. I'd have to find out first whether I have write access to the CVS, though. I don't know what +1 means on other lists, but on this list it means that's such a good idea I'm going to devote my life to implementing it, and I promise to finish it before this time next week -- thanks ;-) Normalization of votes should definitely be standardized across lists. But then, what does implementing a deprecation mean? If it's just adding a deprecation warning, I should be able to make it in time ;o) -- Thomas ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] RE: RE: RE: PersistentMapping
[Tim] ... This worked (which is what I normally do): [build_ext -i] [Thomas] It did for me too. I guess the crucial difference was doing build_ext instead of just build. Yes. Maybe someone with more distutils clue could say something about it; if my guess is right, README.txt should be changed. It's more likely due to people fiddling with zpkg. setup.py build _had_ worked fine for running the tests for years, and I know it still worked fine a few months ago. But I don't normally do that, so didn't notice when it broke. It's not due to changes in ZODB code (no relevant changes have been made there -- or at least none that I made ;-)). For whatever reason, looks like setup.py build in ZODB simply ignores the existence of ZODB's zope/testing directory now (but doesn't ignore ZODB's other zope/* directories ...). ... I don't know what +1 means on other lists, but on this list it means that's such a good idea I'm going to devote my life to implementing it, and I promise to finish it before this time next week -- thanks ;-) Normalization of votes should definitely be standardized across lists. I was pulling your leg -- +1 means the same thing here as elsewhere (strong yes, to the extent that I'll least argue in favor of it). But then, what does implementing a deprecation mean? If it's just adding a deprecation warning, I should be able to make it in time ;o) It means adding the warning, documenting the deprecation in NEWS.txt, adding a test to ensure that the deprecation warning is raised when appropriate, and possibly a bunch of tedious hair to suppress the new warning(s) while the tests that exercise the now-deprecated feature are running (while deprecated, it's still supported until it goes away, so the tests for it still need to run). ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] RE: RE: PersistentMapping
[Tim] ... BTW, setup.py build python test.py should work too; it's a bug that it doesn't (and it worked fine within the last few months), but I don't have time for it now. Well, bug or not, the resident zpkgtools expert tells me he doesn't expect that to work anymore, and that it would take more work than _he_ has time for to try to make it work again. setup.py build is only intended to work from a distribution now, no longer from a checkout. So, README.txt should lose the parts that recommend setup.py build; setup.py build_ext -i is necessary now. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] RE: RE: PersistentMapping
[Julien] Well I'm not sure it works Tim. My comment was to be able to test the build *before* installing the ZODB on the Python. Right, and that's the way ZODB tests are normally run by everyone who works on ZODB (they never _install_ ZODB to run tests). I need to try that again but, if I remember correctly, without the PYTHONPATH update it didn't work out for me. Works for me ;-) A possible difference is that you stuffed ZODB/src at the front of PYTHONPATH, but test.py appends ZODB/src to the end of sys.path. If you have other ZODB or Zope code on sys.path (I never do when I run ZODB tests), your way might be effective at hiding the installed code from ZODB's test.py. I'm not sure. Please figure out whether that's the case in the way you run this stuff. If it is, ZODB's test.py could be changed to prepend ZODB/src instead. BTW, setup.py build python test.py should work too; it's a bug that it doesn't (and it worked fine within the last few months), but I don't have time for it now. right. See later email about that (apparently it's not intended to work anymore -- this is a zpkgtools thing). ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] POSKeyError tracing
Paolo Linux wrote at 2005-11-17 10:23 +0100: ... I suspect that in some cases we do not manage Conflict Error (we retry for 3 times but then we give up...). Could this be connected with POSKeyErrors? It should not. Because giving up should mean you abort the transaction and your transaction should have no effect at all. I cannot reproduce easily the problem so I'd like to know if there are suggestions in order to understand and track down what's happening.. Forgetting aborts or passing persistent objects to different threads may cause POSKeyErrors (though there are probably more possibilities). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Migration of obsolete BTrees to ZOPE 2.8
Tim Peters wrote at 2005-11-18 10:02 -0500: [Margie Watkins CONT] The issue http://www.zope.org/Collectors/Zope/1695 asks for a statement from users who are affected by the missing modules. Unfortunately, that's become out of sight, out of mind since then -- the issue was opened 9 months ago, and to date you're the first to say you really need this (if you read through all the referenced threads, you'll see that everyone else who thought they _might_ need it eventually discovered that they did not). It is no problem as long as you migrate while you are still using a Zope 2.7 (ZODB 3.2) release. We added automatic migration code to our old BTree/Intset using code. But we may need an explicit conversion before we finally upgrade to Zope 2.8 -- to get very rarely used objects upgraded as well. ... I'm not sure to what that refers, but not sure it matters either. There are four threads referenced in the bug report, and AFAICT none of them claim to present a migration method for old BTrees or intSets. The migration is trivial as long as BTree and Intset are still available. We use for example the following BTree migration. # backward (pre Zope 2.8) compatibility try: from BTree import BTree as _BTreeOld # DM 2005-08-25: for unknown reasons, this can raise an AttributeError # rather than an ImportError #except ImportError: _BTreeOld = None except (ImportError, AttributeError): _BTreeOld = None from BTrees.OOBTree import OOBTree as BTree ... tree = self._tree if _BTreeOld is not None and isinstance(tree, _BTreeOld): # still an oldy -- upgrade to modern structure self._tree = BTree(tree) msg = 'BTree upgraded' else: msg = 'BTree was already modern' -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
