Re: [Zope-dev] Mountpoints
Chris McDonough wrote: This merge has been done. Since zopectl test ProductName no longer appears to do the right thing Could you be more specific? cd ~/z2/2/ bin/zopectl test CMFCore Running tests via: /usr/local/bin/python2.3 /home/jim/z2/2/bin/test.py -v --config-file /home/jim/z2/2/etc/zope.conf CMFCore Parsing /home/jim/z2/2/etc/zope.conf Running tests at level 1 Test-module import failures: Module: Products.CMFCore.tests.test_all TypeError: Invalid test_suite, None, in Products.CMFCore.tests.test_all Running unit tests: Running: .. .. .. .. ./home/jim/z2/2/Products/CMFCore/tests/test_PortalContent.py:78: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) # Make sure we have _p_jars .../home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:538: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) ./home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:469: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) /home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:488: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) . ./home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:893: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) ../home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:1119: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) # get a _p_jar for 'sub' ... .. .. Ran 360 tests with 0 failures and 0 errors in 10.558 seconds. Test-modules with import problems: Products.CMFCore.tests.test_all Note that I may have updated the testrunner since you tries this. You might try again. and I can't seem to get test.py to run anything except the entire test suite, That's odd. What did you try? As others have suggested, you should use -h to get help on the many options. I've tried to make the new test runner backward compatible with the old, but some old options are no longer supported. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
On Sat, 2005-10-29 at 17:18 -0400, Jim Fulton wrote: Chris McDonough wrote: This merge has been done. Since zopectl test ProductName no longer appears to do the right thing Could you be more specific? Your checkin after I did the merge fixed the issue... it could not find tests belonging to Products in lib/python/Products if I just provided the Product name when I performed the merge. Now it can: [EMAIL PROTECTED] merge]$ bin/zopectl test ZODBMountPoint Running tests via: /home/chrism/bin/python /home/chrism/projects/zope/merge/bin/test.py -v --config-file /home/chrism/projects/zope/merge/etc/zope.conf ZODBMountPoint Parsing /home/chrism/projects/zope/merge/etc/zope.conf Running tests at level 1 Running unit tests: Running: Ran 4 tests with 0 failures and 0 errors in 0.119 seconds. Thank you! and I can't seem to get test.py to run anything except the entire test suite, That's odd. What did you try? I've since figured this out (with help from -h and Tim, to use -t), and your recent fix actually makes that switch find the ZODBMountPoint tests. So all is well now and I've been checking in test improvements for mounting today. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Chris McDonough wrote: Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)): Check out a fresh, new copy of Zope trunk. Merge your branch into that. Thank you! Eagds. I *thought* I had done just that. I had even printed out your previous handholding email about this before starting. But no. I did not. Instead, to my horror, I realized had *copied* a trunk working copy (of an unknown vintage, although up-to-date according to svn status after an svn up) and then I'd merged the branch into that. So about a minute ago, I followed your instructions literally, figuring that I'd be able to sheepishly move on afterwards, but unfortunately it comes out the same. Literally, here are the commands: $ svn co svn+ssh://svn.zope.org/repos/main/Zope/trunk $ cd trunk $ svn merge -r 38624:HEAD svn+ssh://svn.zope.org/repos/main/Zope/ branches/zodb-blobs-branch $ svn up ... which comes out with Fetching external item into 'lib/python/zope/thread' External at revision 39684. Fetching external item into 'doc/ZEO' A doc/ZEO/cache.txt A doc/ZEO/ZopeREADME.txt A doc/ZEO/README.txt A doc/ZEO/trace.txt A doc/ZEO/howto.txt Updated external to revision 39684. Fetching external item into 'lib/python/zope' svn: Working copy 'lib/python/zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) I am reasonably confident that drinking just one more Yuengling will solve this problem in one way or another. svn:externals suck. A lot. As Tim suggested, you could throw away this check out and start over. A simpler thing you could do is to remove the zope directory and do an svn up. Since we switched to using externals, we see lots of things like this. You just learn to delete the directories in question. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
On Oct 28, 2005, at 9:19 AM, Jim Fulton wrote: svn:externals suck. A lot. As Tim suggested, you could throw away this check out and start over. A simpler thing you could do is to remove the zope directory and do an svn up. That sounds reasonable, but I've done both of those things and no joy yet. I've even switched machines in a hail-mary maybe it's a local client bug idea. Since we switched to using externals, we see lots of things like this. You just learn to delete the directories in question. I wish that worked for me. I'm going to try to offer the directory a cold glass of milk laced with arsenic next. We're getting so cozy with each other, I think it might accept. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Tim Peters] Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)): Check out a fresh, new copy of Zope trunk. Merge your branch into that. [Chris McDonough] Thank you! Eagds. I *thought* I had done just that. I had even printed out your previous handholding email about this before starting. But no. I did not. Instead, to my horror, I realized had *copied* a trunk working copy (of an unknown vintage, although up-to-date according to svn status after an svn up) and then I'd merged the branch into that. So about a minute ago, I followed your instructions literally, figuring that I'd be able to sheepishly move on afterwards, but unfortunately it comes out the same. Literally, here are the commands: $ svn co svn+ssh://svn.zope.org/repos/main/Zope/trunk $ cd trunk $ svn merge -r 38624:HEAD svn+ssh://svn.zope.org/repos/main/Zope/branches/zodb-blobs-branch Didn't you get any output here? I saved mine the last time I tried it, and I expected you'd see the same (except with Linux path separators): U doc C lib\python\Zope2\Startup\datatypes.py U lib\python\Zope2\Startup\zopeschema.xml U lib\python\Products\ZODBMountPoint\tests\testMountPoint.py U lib\python\Products\ZODBMountPoint\MountedObject.py D lib\python\Products\ZODBMountPoint\Mount.py D lib\python\DBTab\DBTab.py D lib\python\DBTab\__init__.py D lib\python\DBTab\CHANGES.txt D lib\python\DBTab\Exceptions.py D lib\python\DBTab\ClassFactories.py D lib\python\DBTab D lib\python\DBTab U lib\python U setup.py U utilities $ svn up ... which comes out with Fetching external item into 'lib/python/zope/thread' External at revision 39684. Fetching external item into 'doc/ZEO' A doc/ZEO/cache.txt A doc/ZEO/ZopeREADME.txt A doc/ZEO/README.txt A doc/ZEO/trace.txt A doc/ZEO/howto.txt Updated external to revision 39684. Fetching external item into 'lib/python/zope' svn: Working copy 'lib/python/zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) Drat. My apologies! Tried again I see that too. Since we _started_ with current Zope trunk here, and your branch doesn't change anything at or under lib/python/zope, I didn't expect that. I was wrong. I am reasonably confident that drinking just one more Yuengling will solve this problem in one way or another. Did it work out? I'm ill today and saw this on my way back to bed, but in 5 minutes of trying I didn't find any combination of rm -rf lib/python/zope and svn cleanup that got me unstuck. Will try again later. [Jim Fulton] svn:externals suck. A lot. As Tim suggested, you could throw away this check out and start over. That's what he tried above -- didn't work for him. Doesn't work for me either, but Chris may have clouded my mind ;-) A simpler thing you could do is to remove the zope directory and do an svn up. He tried that before and reported endless failures. That's why I suggested starting over to begin with. Should have worked ;-) Since we switched to using externals, we see lots of things like this. You just learn to delete the directories in question. That alone wasn't enough for my own Zope trunk checkout -- also needed to do svn cleanup. But the start over from scratch sequence above appears to leave Chris and me in a state where no amount of directory removal and svn cleanup gets rid of Fetching external item into 'lib\python\zope' svn: Working copy 'lib\python\zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) upon the next svn up attempt. Ah, it's the properties on lib/python that are screwing us here! Chris, throw svn revert lib/python into the mix too. That got me unstuck. The problem is that both Jim and I (at least) changed the set of externals listed in lib/python, and SVN absolutely sucks at merging property changes. The revert above gets you back to the externals Zope trunk actually uses today. That's vital because Zope trunk stitches in zope in an entirely different way now. Reverting also stitches in a version of ZODB we don't want to use, but after reverting you can do svn propedit svn:externals lib/python to put back the right version of ZODB for your branch (s/3.4.2/3.6.0b1/g). Alternative: I haven't tried this, but it should work wink too, and would be easier: instead of reverting lib/python, do svn propedit svn:externals lib/python and just delete the part stitching in `zope`; that's this line: zope svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.1-Zope-2.8/src/zope The goal here is to end up with externals on lib/python that do _not_ stitch in zope,
Re: [Zope-dev] Mountpoints
On Fri, 2005-10-28 at 10:12 -0400, Tim Peters wrote: Ah, it's the properties on lib/python that are screwing us here! Chris, throw svn revert lib/python into the mix too. That got me unstuck. The problem is that both Jim and I (at least) changed the set of externals listed in lib/python, and SVN absolutely sucks at merging property changes. The revert above gets you back to the externals Zope trunk actually uses today. That's vital because Zope trunk stitches in zope in an entirely different way now. Reverting also stitches in a version of ZODB we don't want to use, but after reverting you can do svn propedit svn:externals lib/python to put back the right version of ZODB for your branch (s/3.4.2/3.6.0b1/g). Woo hoo! As always, you are my hero, Tim. That worked great. And it as a bonus Zope even *starts* after changing the properties and running svn up. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
This merge has been done. Since zopectl test ProductName no longer appears to do the right thing and I can't seem to get test.py to run anything except the entire test suite, I didn't create any new tests because I wouldn't have had the time to create tests and run them iteratively. That said, all existing tests pass and I did do some interactive stress-testing of the sessioning mount point, both of which made me feel comfortable enough to go ahead and do the merge. Enjoy, - C On Fri, 2005-10-28 at 18:53 -0400, Chris McDonough wrote: On Fri, 2005-10-28 at 10:12 -0400, Tim Peters wrote: Ah, it's the properties on lib/python that are screwing us here! Chris, throw svn revert lib/python into the mix too. That got me unstuck. The problem is that both Jim and I (at least) changed the set of externals listed in lib/python, and SVN absolutely sucks at merging property changes. The revert above gets you back to the externals Zope trunk actually uses today. That's vital because Zope trunk stitches in zope in an entirely different way now. Reverting also stitches in a version of ZODB we don't want to use, but after reverting you can do svn propedit svn:externals lib/python to put back the right version of ZODB for your branch (s/3.4.2/3.6.0b1/g). Woo hoo! As always, you are my hero, Tim. That worked great. And it as a bonus Zope even *starts* after changing the properties and running svn up. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] This merge has been done. Woo hoo! Thank you, Chris! I since stitched in ZODB 3.6.0b2, which is the most recent 3.6 (internal) release. That didn't seem to create any new problems. Since zopectl test ProductName no longer appears to do the right thing Sorry, never used it, don't know anything about it. and I can't seem to get test.py to run anything except the entire test suite, Do test.py -h for help. The -t option will probably help you. For example, $ \python24\python.exe test.py -vv -t test_pop Running tests at level 1 C:\python24\lib\whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) Running unit tests: Running: test_pop_default (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_raises (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_simple (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_raises (AccessControl.tests.testZopeGuards.TestListGuards) test_pop_simple (AccessControl.tests.testZopeGuards.TestListGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestListGuards) Ran 7 tests with 0 failures and 0 errors in 0.000 seconds. $ \python24\python.exe test.py -vv -t pop_val Running tests at level 1 C:\python24\lib\whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) Running unit tests: Running: test_pop_validates (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestListGuards) Ran 2 tests with 0 failures and 0 errors in 0.000 seconds. I didn't create any new tests because I wouldn't have had the time to create tests and run them iteratively. Sorry to ruin your weekend, then ;-) That said, all existing tests pass and I did do some interactive stress-testing of the sessioning mount point, both of which made me feel comfortable enough to go ahead and do the merge. The tests appear to be in exactly as good shape on Windows now as they were before (there are test failures on Windows; opened a Collector issue about them Thursday), so you have a plausible defense in court. I won't sue you if you won't sue me ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
FWIW, I know a couple of people are depending on this, so here's an update. I am working on merging multidatabase support, but I'm having some merge/update troubles (if you're interested in the symptoms, see http://www.plope.com/Members/chrism/heres_to_cvs). I suspect I'll work it out, but I've got my nose in Subversion documentation at the moment. On Oct 27, 2005, at 1:09 AM, Chris McDonough wrote: I lied. Due to completely preventable circumstances, this merge won't be done tonight; instead, it will be done tomorrow evening. - C On Mon, 2005-10-24 at 16:41 -0400, Tim Peters wrote: [Chris McDonough] Thanks for this! Not required, so long as I get to thank you for finishing it ;-) Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand. I'm sure that failure will go away by itself when you're working on the trunk instead of the branch. What I'd do now: - Check out Zope trunk. - Merge the branch into your trunk sandbox, and forget the branch. - Fix merge conflicts. I got one, in datatypes.py, and I didn't know immediately what to do about it so stopped there. You'll have better luck ;-). Note that, under SVN, after you fix a conflict, you have to do svn resolved path/to/conflicted/file; that's a gimmick to make sure you don't forget about conflicts. - svn up to make sure you've got all the externals the merged files point at. - svn up from time to time thereafter, to suck in other trunk changes as they get made. - Check it in when it's stable. - If it takes longer than expected, make a _new_ branch _from_ your merged-into-trunk local trunk sandbox. (That's easy: make a branch directory, svn switch to it from your local merged trunk sandbox, and svn commit -- all done). It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge. As above, you can do a _local_ merge right away. This would save you from other decoys (like the DeprecationWarnings that would no longer exist if you were using the trunk instead of the brach, and the failing-on-branch-but-not-trunk Five test). I recall that, historically, the Zope tests never failed when Zope mounting was in fact broken, so a fat +1 to beefing test coverage there. I have this on my plate for Wednesday evening. Understood; there really isn't any good TV on Wednesdays anymore ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Tim, trying to comfort a suffering Chris] ... End of story. Unless you feel you need to make another branch. In that case, still do the two steps above first. Then create a new branch from Zope trunk, svn switch your merged sandbox to that new branch, then svn checkin. That last part should have said svn commit. If it had, you'd already be done ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)): Check out a fresh, new copy of Zope trunk. Merge your branch into that. Thank you! Eagds. I *thought* I had done just that. I had even printed out your previous handholding email about this before starting. But no. I did not. Instead, to my horror, I realized had *copied* a trunk working copy (of an unknown vintage, although up-to-date according to svn status after an svn up) and then I'd merged the branch into that. So about a minute ago, I followed your instructions literally, figuring that I'd be able to sheepishly move on afterwards, but unfortunately it comes out the same. Literally, here are the commands: $ svn co svn+ssh://svn.zope.org/repos/main/Zope/trunk $ cd trunk $ svn merge -r 38624:HEAD svn+ssh://svn.zope.org/repos/main/Zope/ branches/zodb-blobs-branch $ svn up ... which comes out with Fetching external item into 'lib/python/zope/thread' External at revision 39684. Fetching external item into 'doc/ZEO' A doc/ZEO/cache.txt A doc/ZEO/ZopeREADME.txt A doc/ZEO/README.txt A doc/ZEO/trace.txt A doc/ZEO/howto.txt Updated external to revision 39684. Fetching external item into 'lib/python/zope' svn: Working copy 'lib/python/zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) I am reasonably confident that drinking just one more Yuengling will solve this problem in one way or another. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] FWIW, I know a couple of people are depending on this, so here's an update. I am working on merging multidatabase support, but I'm having some merge/update troubles (if you're interested in the symptoms, see http://www.plope.com/Members/chrism/heres_to_cvs). I suspect I'll work it out, but I've got my nose in Subversion documentation at the moment. Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)): Check out a fresh, new copy of Zope trunk. Merge your branch into that. End of story. Unless you feel you need to make another branch. In that case, still do the two steps above first. Then create a new branch from Zope trunk, svn switch your merged sandbox to that new branch, then svn checkin. This way you shouldn't have any of the problems you've been seeing. If you do, double your money back ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
I lied. Due to completely preventable circumstances, this merge won't be done tonight; instead, it will be done tomorrow evening. - C On Mon, 2005-10-24 at 16:41 -0400, Tim Peters wrote: [Chris McDonough] Thanks for this! Not required, so long as I get to thank you for finishing it ;-) Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand. I'm sure that failure will go away by itself when you're working on the trunk instead of the branch. What I'd do now: - Check out Zope trunk. - Merge the branch into your trunk sandbox, and forget the branch. - Fix merge conflicts. I got one, in datatypes.py, and I didn't know immediately what to do about it so stopped there. You'll have better luck ;-). Note that, under SVN, after you fix a conflict, you have to do svn resolved path/to/conflicted/file; that's a gimmick to make sure you don't forget about conflicts. - svn up to make sure you've got all the externals the merged files point at. - svn up from time to time thereafter, to suck in other trunk changes as they get made. - Check it in when it's stable. - If it takes longer than expected, make a _new_ branch _from_ your merged-into-trunk local trunk sandbox. (That's easy: make a branch directory, svn switch to it from your local merged trunk sandbox, and svn commit -- all done). It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge. As above, you can do a _local_ merge right away. This would save you from other decoys (like the DeprecationWarnings that would no longer exist if you were using the trunk instead of the brach, and the failing-on-branch-but-not-trunk Five test). I recall that, historically, the Zope tests never failed when Zope mounting was in fact broken, so a fat +1 to beefing test coverage there. I have this on my plate for Wednesday evening. Understood; there really isn't any good TV on Wednesdays anymore ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] There is a wrinkle about performing this merge that eluded my memory until now. To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code databases and database_name options that may now be passed into ZODB.DB's constructor: http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=38626r1=38574r2=38626 The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. ... Chris, here's the current state: - The following is on current ZODB trunk, and on the new tag svn://svn.zope.org/repos/main/ZODB/tags/3.6.0b1 I suggest changing the zodb-blobs-branch to use that tag now (and merging that branch to Zope trunk when it's all happy again). - I added an optional new database_name key to zodb config. I understand that you may not want to use it in Zope 2.9, taking the database name from Zope's zodb_db section instead. That's fine. I expect that whatever name (however decided) should be used can be poked into config.database_name before calling ZODBDatabase.open(). Should check that the section name and database_name key aren't both specified? Probably, ya. - ZODBDatabase.open() has an optional new databases= argument, so that part's still exactly the same way you did it (except that I didn't add that argument to BaseConfig.open() too -- I don't think it belongs there, as BaseConfig is also a base class for classes whose open() overrides don't support a databases= argument; the ZODBDatabase subclass _extends_ BaseConfig's open() in this respect). Is there more I can do to help this along (or, perhaps, delay it more ;-))? If so, just ask. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Chris, FYI, I stitched ZODB 3.6.0b1 into zodb-blobs-branch, and changed ZopeDatabase.createDB() to plug database_name into config instead of passing it to ZODBDatabase.open(). The checkin msg summarizes test results; since I haven't work on this branch before, I'm not sure what was expected here (I didn't get a clean test run before stitching 3.6.0b1 in either): http://mail.zope.org/pipermail/zope-checkins/2005-October/029904.html How would you like to proceed? ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Thanks for this! Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand. It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge. I have this on my plate for Wednesday evening. - C On Oct 24, 2005, at 2:45 PM, Tim Peters wrote: Chris, FYI, I stitched ZODB 3.6.0b1 into zodb-blobs-branch, and changed ZopeDatabase.createDB() to plug database_name into config instead of passing it to ZODBDatabase.open(). The checkin msg summarizes test results; since I haven't work on this branch before, I'm not sure what was expected here (I didn't get a clean test run before stitching 3.6.0b1 in either): http://mail.zope.org/pipermail/zope-checkins/2005-October/ 029904.html How would you like to proceed? ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] Thanks for this! Not required, so long as I get to thank you for finishing it ;-) Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand. I'm sure that failure will go away by itself when you're working on the trunk instead of the branch. What I'd do now: - Check out Zope trunk. - Merge the branch into your trunk sandbox, and forget the branch. - Fix merge conflicts. I got one, in datatypes.py, and I didn't know immediately what to do about it so stopped there. You'll have better luck ;-). Note that, under SVN, after you fix a conflict, you have to do svn resolved path/to/conflicted/file; that's a gimmick to make sure you don't forget about conflicts. - svn up to make sure you've got all the externals the merged files point at. - svn up from time to time thereafter, to suck in other trunk changes as they get made. - Check it in when it's stable. - If it takes longer than expected, make a _new_ branch _from_ your merged-into-trunk local trunk sandbox. (That's easy: make a branch directory, svn switch to it from your local merged trunk sandbox, and svn commit -- all done). It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge. As above, you can do a _local_ merge right away. This would save you from other decoys (like the DeprecationWarnings that would no longer exist if you were using the trunk instead of the brach, and the failing-on-branch-but-not-trunk Five test). I recall that, historically, the Zope tests never failed when Zope mounting was in fact broken, so a fat +1 to beefing test coverage there. I have this on my plate for Wednesday evening. Understood; there really isn't any good TV on Wednesdays anymore ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Tim Peters wrote: [Chris McDonough] There is a wrinkle about performing this merge that eluded my memory until now. To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code databases and database_name options that may now be passed into ZODB.DB's constructor: http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=38626r1=38574r2=38626 The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. In case you're interested, the code that actually makes use of this feature in the zodb-blobs-branch is in the Zope2.datatypes.DBTab.getDatabase method. Is this change acceptable for a merge into the ZODB HEAD? Turns out that a release of Zope3 has already been made that supports multidatabases, and I'd naturally prefer to follow the lead of a Zope that's already out there. Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that? Do you mean an example of a zope.conf that uses it? From a customer engagement: zodb main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb a filestorage path $DATADIR/A.fs /filestorage /zodb We decided to use the section names for the database names. This was to avoid changing ZODB. I'm not sure that that was a good idea. This approach has two disadvantages: - Because section names are case insenstive, database names end up being lower case, whether we want them to be or not. - It may not be obvious that the section name is also the database name. I'm really unsure about whether this is a disadvantage. I'm not sure if: zodb name main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb name a filestorage path $DATADIR/A.fs /filestorage /zodb is better or worse than the first version. I'm inclined to think that any time you have sections of the same type, it is desireable to give them names, in which case we might be tempted to list the names twice. The Zope3 code in question is in src/zope/app/appsetup/appsetup.py function multi_database(). Note that they didn't change any ZODB files, instead they give values to a DB's .databases and .database_name attributes after constructing the DB. While that might be questionable in general cough, the implementation of multidatabases was meant to be both concrete and public. It's not an accident that ZODB's tutorial tests/multidb.txt doctest explains and exploits details of the concrete implementation -- it's not meant to be abstract. IOW, poking in new values for these attributes isn't considered to be evil. I'd be happy to plumb this through the factories open method. It would seem to me that we only need to be able to pass a databases argument. The factory presumably knows it's own name. It could then pass the databases dict and the name to the DB constructor. I believe (here's where the example I can't find would nail it) they use the name on a zodb section as the DB's database_name. Fred points out that ZConfig section names are case-insensitive, forced to lowercase, so that zodb CHRIS and zodb cHris have the same name. That's not ideal, and threading these attributes throughout ZODB's config.py instead (as you did) would be a sane way to worm around that. I haven't looked at Chris's changes. I was pretty happy that the changes we made in Z3 were fairly localized and small. But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense. OTOH, this feature has hardly been used in Z3. I added to ZODB because I had been meaning to for some time ad because we needed it for a customer. I don't think anyone else has used it, so I don't think there's much established pattern in Z3. Then again, I'm not sure, except for the case insentitivity issue that we didn't do it the best way. I'd much rather revisit the case insenstitivity of section names in ZConfig. I think that if ZConfig section names were case sensitive or at least case preserving, I'd be happy with the approach we took. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Tim Peters, on multidatabase support in Zope3] ... Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that? [Jim Fulton] Do you mean an example of a zope.conf that uses it? Yes, that's all -- just a concrete example. I could have guessed one (and did later ;-)), but guesses can be wrong. From a customer engagement: zodb main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb a filestorage path $DATADIR/A.fs /filestorage /zodb Thanks! That did the trick. We decided to use the section names for the database names. This was to avoid changing ZODB. I'm not sure that that was a good idea. Let's change it. This approach has two disadvantages: - Because section names are case insenstive, database names end up being lower case, whether we want them to be or not. - It may not be obvious that the section name is also the database name. It isn't obvious -- and unlike for other zodb keys, a user can't look in ZODB's component.xml now to find out anything about this usage. They can, e.g., look there to see that they can specify cache-size, that it's an integer, and that it defaults to 5000. They can also find helpful description sections for many keys. Using the section name is pure out-of-the-blue magic in contrast. I'm really unsure about whether this is a disadvantage. I'm not sure if: zodb name main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb name a filestorage path $DATADIR/A.fs /filestorage /zodb is better or worse than the first version. I think it's worse, but mostly because a key with name name is also an option in _related_ sections, but with unrelated meaning. For example, if you had a nested zeoclient section there it could also have specified a name key, which would have nothing to do with the zodb key named name. Nesting options with the same name gets confusing quickly. OTOH, I would like the explicit key better if it had a different name, say zodb multidb-name main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb multidb-name a filestorage path $DATADIR/A.fs /filestorage /zodb I'm inclined to think that any time you have sections of the same type, it is desireable to give them names, in which case we might be tempted to list the names twice. Sounds orthogonal to me. If it's desirable that whenever multiple sections of the same type appear, they must be given names, that's fine, but the way to enforce or encourage that isn't to make all sections that may appear more than once give some _meaning_ to the section name. It was just expedient in this specific case, right? ... I'd be happy to plumb this through the factories open method. It would seem to me that we only need to be able to pass a databases argument. Right. The factory presumably knows it's own name. It could then pass the databases dict and the name to the DB constructor. If I change the zodb config to say there's a new optional key for the multidatabase name (like the multidb-name key in the made-up example above), then the factory will have the same access to that as it has for other existing ZConfig-specified keys (like cache-size). BTW, I think there's a related buglet in Zope3's multi_database(): name = factory.name or '' ... db.database_name = name Defaulting to an empty string for the name is really a bit of abuse, since the documented default database_name for ZODB.DB is unnamed, and I doubt Zope3 documented that it changed this default ;-). An explicit ZConfig key here would supply that correct default. ... I haven't looked at Chris's changes. I was pretty happy that the changes we made in Z3 were fairly localized and small. Adding the optional new key to the zodb config, and threading the `databases` arg thru ZODB's config.py, are also small changes. But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense. OTOH, this feature has hardly been used in Z3. I added to ZODB because I had been meaning to for some time ad because we needed it for a customer. I don't think anyone else has used it, so I don't think there's much established pattern in Z3. Then again, I'm not sure, except for the case insentitivity issue that we didn't do it the best way. I'd much rather revisit the case insenstitivity of section names in ZConfig. I think that if ZConfig section names were case sensitive or at least case preserving, I'd be happy with the approach we took. Note that if we add an explicit new key, the case issue goes away (for this specific
Re: [Zope-dev] Mountpoints
Tim Peters wrote: [Chris McDonough] I think I may need some remedial SVN help because I don't want to do this in a stupid way. Hopefully someone will be willing to guide me through this. I'll be in FB tomorrow if you'd like to pair on it (while in theory Jim might object, I think he thinks getting this done is important enough to offer real help -- especially if he doesn't have to offer it himself wink). Works for me. :) Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
... [Chris McDonough] Thanks for the offer! I won't be able to visit ZC world HQ tomorrow, though unless you'd be there and willing to start around 10pm. Alas, they're still under the delusion that 10 _am_ is late here, so while I agree 10pm is saner on all counts, I'll be gone before then. Other duties is my official excuse but I'm also horrified by the idea that I'd be expected to wear pants if I came over there. That's just uncivilized. :-) There was such a backlash against the pants required policy that it's OK to wear diapers instead now. Progress, anyway. ... [Tim] Check. Question: does zodb-blobs-branch contain anything you _don't_ want to see on Zope trunk now? You didn't mention anything like that here. No (save for inappropriate svn:externals to ZODB and ZEO). In that case, and since you say later there's no reason to keep this branch after the merge is done, I suggest merging directly from zodb-blobs-branch to Zope trunk. Unless I'm missing something, there really doesn't seem to be a point to creating another intermediate branch first. ... Yes. The zodb-blobs-branch can just die after this merge if there's a way to get delete branches entirely. Yes and no ;-) A branch or a tag in SVN is just another named directory, with non-mandatory conventions for choosing its path name. It can be deleted, like any other directory. That doesn't get rid of it entirely, just as no directory can be gotten rid of entirely: you can always revert the checkin in which it was deleted, and then it will magically reappear. Unlike as under CVS, though, deleted branches don't keep punching you in the face if you don't go out of your way to find them. For example, these are all the visible ZODB branches today: $ svn list svn://svn.zope.org/repos/main/ZODB/branches 3.3/ 3.4/ 3.5/ blob-merge-branch/ ctheune-blobsupport/ Note that none of the other branches we created at the ZODB sprint, or most of the branches created since then, still show up (I deleted them after merging to then-current ZODB trunk). SVN is very nice this way! If there is to be a long-lived branch, it will be the blob-merge-branch of ZODB. Afraid so, yes. Is it time to delete the ctheune-blobsupport branch? Given that Zope 2.9 is not going to ship with blob support due to feature freeze, I think this means that we have until May to allow the blob-merge-branch to get utterly out of sync with the ZODB trunk. We can then easily wait until, say, the last week in April to worry about issues caused by that desynchronization. The work necessary to remerge should provide just the appropriate amount of delay to allow blobs to miss the next major Zope release. ;-) Excellent! I'm glad you're thinking hard about this -- it gets hard delaying release after release all by myself ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] There is a wrinkle about performing this merge that eluded my memory until now. To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code databases and database_name options that may now be passed into ZODB.DB's constructor: http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=38626r1=38574r2=38626 The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. In case you're interested, the code that actually makes use of this feature in the zodb-blobs-branch is in the Zope2.datatypes.DBTab.getDatabase method. Is this change acceptable for a merge into the ZODB HEAD? Turns out that a release of Zope3 has already been made that supports multidatabases, and I'd naturally prefer to follow the lead of a Zope that's already out there. Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that? The Zope3 code in question is in src/zope/app/appsetup/appsetup.py function multi_database(). Note that they didn't change any ZODB files, instead they give values to a DB's .databases and .database_name attributes after constructing the DB. While that might be questionable in general cough, the implementation of multidatabases was meant to be both concrete and public. It's not an accident that ZODB's tutorial tests/multidb.txt doctest explains and exploits details of the concrete implementation -- it's not meant to be abstract. IOW, poking in new values for these attributes isn't considered to be evil. I believe (here's where the example I can't find would nail it) they use the name on a zodb section as the DB's database_name. Fred points out that ZConfig section names are case-insensitive, forced to lowercase, so that zodb CHRIS and zodb cHris have the same name. That's not ideal, and threading these attributes throughout ZODB's config.py instead (as you did) would be a sane way to worm around that. But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense. Some mechanics: if we do need to make changes, ya, ZODB trunk is the place to do it. Work on the branch should use ZODB trunk now. When that's as ready to go as it's going to get, let me know and I'll make an internal release of ZODB 3.6 so we can use a ZODB tag before merging into Zope trunk. (An internal release just means I update ZODB's NEWS.txt, fiddle version numbers and dates in umpteen places, and make a tag so other projects can use that -- it's the tag that's important here; an internal release does not involve making tarballs, Windows installers, announcements (etc), so it's much cheaper and goes much faster (minutes vs hours) than making a public release.) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Mountpoints
I was just reminded (by a zope3-dev question) of mount points. Zope2 has a mount-point capability that requires baboon patching of ZODB. The ZODB that will be used for Zope 2.9 has a multi-database feature that would allow a much simpler Zope 2 mount implementation that would not require even monkey patching of ZODB. It also, I think, would provide much more robust mounting behavior. It would be great if someone would update mounting for Zoep 2.9 to use multi-databases. I'd be happy to provide advice for such an effort. Of course, this would need to be completed before the November 1 feature freeze. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). November 1 feature freeze, eh? I'd love to get blobs in before this too but I don't know if it will be possible. If not, it would really suck. On Tue, 2005-10-18 at 10:22 -0400, Jim Fulton wrote: I was just reminded (by a zope3-dev question) of mount points. Zope2 has a mount-point capability that requires baboon patching of ZODB. The ZODB that will be used for Zope 2.9 has a multi-database feature that would allow a much simpler Zope 2 mount implementation that would not require even monkey patching of ZODB. It also, I think, would provide much more robust mounting behavior. It would be great if someone would update mounting for Zoep 2.9 to use multi-databases. I'd be happy to provide advice for such an effort. Of course, this would need to be completed before the November 1 feature freeze. Jim ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Chris McDonough wrote: This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. November 1 feature freeze, eh? Yup. :) I'd love to get blobs in before this too but I don't know if it will be possible. I don't think so. Nov 1 is less than 2 weeks away. I think we need to be realistic. I would definately want to review this before it got merged into the head. I think Tim might too. I don't think it's ready for my review yet. Anyway, if you want to make this a priority, I'm willing to make time for review as best I can. If you want to try to finish this before Nov 1, I'll try to help by doing timely reviews. Note that this will be of limited usefulness without also updating Zope file implementations (z2 and z3) to take advantage of it, although people could build add-on packages that used it. If not, it would really suck. Well, I dunno, if it's not ready, June is not that far away. I'm confident that we could get this done by the May 1 feature freeze if this was a priority. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
On Tue, 2005-10-18 at 11:32 -0400, Jim Fulton wrote: Chris McDonough wrote: This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. OK, I will do so. November 1 feature freeze, eh? Yup. :) I'd love to get blobs in before this too but I don't know if it will be possible. I don't think so. Nov 1 is less than 2 weeks away. I think we need to be realistic. Yep. We'll wait on it then. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). [Jim Fulton] Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. FYI, Zope trunk (in SVN) is current 2.9 development. [Chris] OK, I will do so. In that case, I'm willing to share wink this set of probably-related open Collector issues: ZODBMountPoint should not monkey-patch ZODB http://www.zope.org/Collectors/Zope/1525 Clean up mounts, TemporaryFolder http://www.zope.org/Collectors/Zope/1800 ZODB: Mounting broken for non default transaction manager http://www.zope.org/Collectors/Zope/1875 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). [Jim Fulton] Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. FYI, there's a problem with that: Zope trunk (2.9) is still using ZODB 3.4, and multi-databases weren't introduced before ZODB 3.5. The _intent_ has been that 2.9 would use ZODB 3.5 or even 3.6, but that got hung up due to problems with switching the 5 integration part to use zpkgtools. I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the 5 guys to do something before trying again. Sorry for not being more specific ... ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[It doesn't look like my response went to the zope-dev list; re-sending.] On Tuesday 18 October 2005 15:43, Tim Peters wrote: I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the 5 guys to do something before trying again. Sorry for not being more specific ... We need to do the zpkg/ZODB switch all at once because it affects how extensions get their include files. When I tried switching the Zope 2 trunk before, there was a problem due to Five tests failing. I don't remember the details, but the Five-folks seemed to think things would be better with newer versions of Five. Since Five development is done elsewhere, though, it's hard to tell what the deal is with that. I snapshotted what I did get done as the zpkg-build-branch, so we can get back to it without duplicating work. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Society attacks early, when the individual is helpless. --B.F. Skinner ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Five status on Zope trunk (was Re: [Zope-dev] Mountpoints)
[Tim] I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the 5 guys to do something before trying again. Sorry for not being more specific ... [Fred] We need to do the zpkg/ZODB switch all at once because it affects how extensions get their include files. Aha! That vaguely rings another vague bell ;-) The ZODB 3.5+ source code spells #include in its C files in a different way, and that can't work with the way Zope trunk is currently built. When I tried switching the Zope 2 trunk before, there was a problem due to Five tests failing. I don't remember the details, but the Five-folks seemed to think things would be better with newer versions of Five. Since Five development is done elsewhere, though, it's hard to tell what the deal is with that. Can anyone working on Five comment here, please? I gather that the first 2.9 beta release occurs in 2 weeks, and if Fred can't do his zpkgtools work, I can't update the trunk to a newer ZODB, and then Chris McDonough can't do his rework of mount points (the original topic of this thread), and then (best case) the world economy collapses. I snapshotted what I did get done as the zpkg-build-branch, so we can get back to it without duplicating work. So there's no reason yet to be depressed ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Note that I wormed my way around the new ZODB compilation issues by changing the setup.py file on the zodb-blobs-branch (a minor tweak of the Zope HEAD).. here's the comment I left to myself. # added . to EXTENSIONCLASS_INCLUDEDIRS in order to be able to compile # ZODB HEAD code (which uses qualified paths to find header files). # This # should be a temporary change, thrown away once we use zpkg to package # Zope. EXTENSIONCLASS_INCLUDEDIRS = ['ExtensionClass', '.'] - C On Tue, 2005-10-18 at 16:18 -0400, Fred Drake wrote: [It doesn't look like my response went to the zope-dev list; re-sending.] On Tuesday 18 October 2005 15:43, Tim Peters wrote: I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the 5 guys to do something before trying again. Sorry for not being more specific ... We need to do the zpkg/ZODB switch all at once because it affects how extensions get their include files. When I tried switching the Zope 2 trunk before, there was a problem due to Five tests failing. I don't remember the details, but the Five-folks seemed to think things would be better with newer versions of Five. Since Five development is done elsewhere, though, it's hard to tell what the deal is with that. I snapshotted what I did get done as the zpkg-build-branch, so we can get back to it without duplicating work. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Society attacks early, when the individual is helpless. --B.F. Skinner ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
On Tue, 2005-10-18 at 11:32 -0400, Jim Fulton wrote: Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. I think I may need some remedial SVN help because I don't want to do this in a stupid way. Hopefully someone will be willing to guide me through this. There exists a branch of the Zope 2 trunk named the zodb-blobs-branch. It was created as a branch of the Zope 2 trunk on September 25 (about 3 weeks ago). The zodb-blobs-branch contains code that use Tim's multidatabase support for mountpoint support rather than the older DBTab code which monkeypatched ZODB and did other nasty things. It contains a few other ancillary changes that make it possible to use a HEAD-ish ZODB package in Zope 2 as well (including changes to the setup.py I mentioned which allows a newer ZODB to be compiled). The zodb-blobs-branch happens to link in an svn external for the ZODB package which currently points to a *ZODB* branch named blob-merge-branch. This is really the only thing blob-ish about the zodb-blobs-branch. All of the changes that actually cause blobs to be supported are isolated in that ZODB branch. The rest of the changes are really just to support a later ZODB revision within Zope. It's likely that this svn external can be changed to point to any 3.6-ish ZODB branch without breaking things too badly. So I'd *think* I'd like to: - create a SVN branch *from the zodb-blobs-branch* named mountpoint-merge-branch - change the svn external for lib/python/ZODB on the mountpoint-merge-branch to point at the proper ZODB revision - merge the changes that have happened since September 25 on the Zope 2 trunk into the mountpoint-merge-branch - merge the mountpoint-merge-branch back in to the trunk. But I've read the Zope SVN FAQ and it frowns on the practice of merging trunk changes into a branch and back again. Is there a better way to do this other than applying the changes manually to the HEAD? Also, what is the right branch of ZODB to use in the svn:external for lib/python/ZODB so I can test that it all works ok before I actually perform the merge to the HEAD? - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
On Tue, 2005-10-18 at 22:21 -0400, Tim Peters wrote: [Chris McDonough] I think I may need some remedial SVN help because I don't want to do this in a stupid way. Hopefully someone will be willing to guide me through this. I'll be in FB tomorrow if you'd like to pair on it (while in theory Jim might object, I think he thinks getting this done is important enough to offer real help -- especially if he doesn't have to offer it himself wink). Thanks for the offer! I won't be able to visit ZC world HQ tomorrow, though unless you'd be there and willing to start around 10pm. Other duties is my official excuse but I'm also horrified by the idea that I'd be expected to wear pants if I came over there. That's just uncivilized. :-) The zodb-blobs-branch contains code that use Tim's multidatabase support for mountpoint support rather than the older DBTab code which monkeypatched ZODB and did other nasty things. It contains a few other ancillary changes that make it possible to use a HEAD-ish ZODB package in Zope 2 as well (including changes to the setup.py I mentioned which allows a newer ZODB to be compiled). Check. Question: does zodb-blobs-branch contain anything you _don't_ want to see on Zope trunk now? You didn't mention anything like that here. No (save for inappropriate svn:externals to ZODB and ZEO). - merge the changes that have happened since September 25 on the Zope 2 trunk into the mountpoint-merge-branch Why? It may create headaches and I don't see the attraction. Have people been checking in changes to the same files over the last 3 weeks? Even if they have, conflicts are probably easier to deal with when the new branch gets merged back to the trunk. No, people haven't been changing the same files (except for maybe setup.py) so seems like good advice. This is really what I needed to understand. But I've read the Zope SVN FAQ and it frowns on the practice of merging trunk changes into a branch and back again. That's because it's so easy to lose track of what you're doing then. It probably can't be avoided on very long-lived branches, but this branch is only several weeks old now, right? Yes. The zodb-blobs-branch can just die after this merge if there's a way to get delete branches entirely. If there is to be a long-lived branch, it will be the blob-merge-branch of ZODB. Given that Zope 2.9 is not going to ship with blob support due to feature freeze, I think this means that we have until May to allow the blob-merge-branch to get utterly out of sync with the ZODB trunk. We can then easily wait until, say, the last week in April to worry about issues caused by that desynchronization. The work necessary to remerge should provide just the appropriate amount of delay to allow blobs to miss the next major Zope release. ;-) Also, what is the right branch of ZODB to use in the svn:external for lib/python/ZODB so I can test that it all works ok before I actually perform the merge to the HEAD? I never leave a Zope pointing at a ZODB branch -- only at a ZODB tag. The only two suitable tags at this time are ZODB/tags/3.5.1 and ZODB/tags/3.6.0a4 Either should work fine for you. If you use the latter, it may save me some time later ;-) But in either case, it won't last long (I'll have to stitch in a 3.5.2 beta or 3.6.0 beta soon anyway, and tags for those don't exist now). Great, that's what I needed to know. Thanks Tim! - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
There is a wrinkle about performing this merge that eluded my memory until now. To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code databases and database_name options that may now be passed into ZODB.DB's constructor: http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=38626r1=38574r2=38626 The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. In case you're interested, the code that actually makes use of this feature in the zodb-blobs-branch is in the Zope2.datatypes.DBTab.getDatabase method. Is this change acceptable for a merge into the ZODB HEAD? - C On Wed, 2005-10-19 at 01:02 -0400, Chris McDonough wrote: On Tue, 2005-10-18 at 22:21 -0400, Tim Peters wrote: [Chris McDonough] I think I may need some remedial SVN help because I don't want to do this in a stupid way. Hopefully someone will be willing to guide me through this. I'll be in FB tomorrow if you'd like to pair on it (while in theory Jim might object, I think he thinks getting this done is important enough to offer real help -- especially if he doesn't have to offer it himself wink). Thanks for the offer! I won't be able to visit ZC world HQ tomorrow, though unless you'd be there and willing to start around 10pm. Other duties is my official excuse but I'm also horrified by the idea that I'd be expected to wear pants if I came over there. That's just uncivilized. :-) The zodb-blobs-branch contains code that use Tim's multidatabase support for mountpoint support rather than the older DBTab code which monkeypatched ZODB and did other nasty things. It contains a few other ancillary changes that make it possible to use a HEAD-ish ZODB package in Zope 2 as well (including changes to the setup.py I mentioned which allows a newer ZODB to be compiled). Check. Question: does zodb-blobs-branch contain anything you _don't_ want to see on Zope trunk now? You didn't mention anything like that here. No (save for inappropriate svn:externals to ZODB and ZEO). - merge the changes that have happened since September 25 on the Zope 2 trunk into the mountpoint-merge-branch Why? It may create headaches and I don't see the attraction. Have people been checking in changes to the same files over the last 3 weeks? Even if they have, conflicts are probably easier to deal with when the new branch gets merged back to the trunk. No, people haven't been changing the same files (except for maybe setup.py) so seems like good advice. This is really what I needed to understand. But I've read the Zope SVN FAQ and it frowns on the practice of merging trunk changes into a branch and back again. That's because it's so easy to lose track of what you're doing then. It probably can't be avoided on very long-lived branches, but this branch is only several weeks old now, right? Yes. The zodb-blobs-branch can just die after this merge if there's a way to get delete branches entirely. If there is to be a long-lived branch, it will be the blob-merge-branch of ZODB. Given that Zope 2.9 is not going to ship with blob support due to feature freeze, I think this means that we have until May to allow the blob-merge-branch to get utterly out of sync with the ZODB trunk. We can then easily wait until, say, the last week in April to worry about issues caused by that desynchronization. The work necessary to remerge should provide just the appropriate amount of delay to allow blobs to miss the next major Zope release. ;-) Also, what is the right branch of ZODB to use in the svn:external for lib/python/ZODB so I can test that it all works ok before I actually perform the merge to the HEAD? I never leave a Zope pointing at a ZODB branch -- only at a ZODB tag. The only two suitable tags at this time are ZODB/tags/3.5.1 and ZODB/tags/3.6.0a4 Either should work fine for you. If you use the latter, it may save me some time later ;-) But in either case, it won't last long (I'll have to stitch in a 3.5.2 beta or 3.6.0 beta soon anyway, and tags for those don't exist now). Great, that's what I needed to know. Thanks Tim! - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
