Re: [ZODB-Dev] what could cause this kind of cache corruption?
Christian Theune wrote: However, I was hoping for inclusion of at least the `storage-iterator-branch` in 3.9 and I asked a few times over the past months already for review but haven't gotten any responses at all so far. This branch has been used in production for a few month already. In addition there's the `bushy-directory` branch which introduces a new blob directory structure to avoid scalability issues with filesystems like ext2/3 that only accept limited numbers of entries in a directory. This branch is being used in production but needs a little which should happen in the next days. (The packing is reported to block.) At last there's a small `ctheune-betterimport` branch which fixes ZEXP import for weakref objects and has been sitting around for almost a year now too. Any joy on getting these merged? How about getting ClientStorage fixed so zeoraid works properly? ;-) cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
Hi, On Mon, 2008-08-11 at 09:37 -0400, Jim Fulton wrote: On Aug 3, 2008, at 4:03 AM, Christian Theune wrote: On Fri, 2008-08-01 at 09:21 -0400, Jim Fulton wrote: On Jul 31, 2008, at 1:53 PM, Chris Withers wrote: What I'd *really* like is a stable zodb release with Christian's patches for zeoraid and Shane's patches for RelStorage that then feeds through into a stable release of Zope 2. I'm not familiar with Christians patches. If they are bug fixes, they could make it into 3.8. I plan to include Shane's patches in 3.9. I'm not sure when there will be a stable release of 3.9. My patches aren't bug fixes and the plan was never to include them to ZODB 3.8. However, I was hoping for inclusion of at least the `storage-iterator-branch` in 3.9 and I asked a few times over the past months already for review but haven't gotten any responses at all so far. This branch has been used in production for a few month already. I haven' t had time to review this yet. I'll get to it eventually. I'm sure I'll get to it before 3.9 is released. \o/ In addition there's the `bushy-directory` branch which introduces a new blob directory structure to avoid scalability issues with filesystems like ext2/3 that only accept limited numbers of entries in a directory. Hm, this sounds like a bug fix if it prevents failures. It is. We ran into a failure on a production system that ran ext(2|3). This branch is being used in production but needs a little which should happen in the next days. (The packing is reported to block.) Is the packing blockage unique to this branch? Blocking currently happens on both trunk and 3.8. At last there's a small `ctheune-betterimport` branch which fixes ZEXP import for weakref objects and has been sitting around for almost a year now too. Is this a bug fix for 3.8? Or a feature for 3.9? I can't remember when 3.8 came out. Back than it was a feature. OTOH you could call it a bugfix because you could create ZEXP files that could never be imported again. Christian -- Christian Theune · [EMAIL PROTECTED] gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Fri, 2008-08-01 at 09:21 -0400, Jim Fulton wrote: On Jul 31, 2008, at 1:53 PM, Chris Withers wrote: What I'd *really* like is a stable zodb release with Christian's patches for zeoraid and Shane's patches for RelStorage that then feeds through into a stable release of Zope 2. I'm not familiar with Christians patches. If they are bug fixes, they could make it into 3.8. I plan to include Shane's patches in 3.9. I'm not sure when there will be a stable release of 3.9. My patches aren't bug fixes and the plan was never to include them to ZODB 3.8. However, I was hoping for inclusion of at least the `storage-iterator-branch` in 3.9 and I asked a few times over the past months already for review but haven't gotten any responses at all so far. This branch has been used in production for a few month already. In addition there's the `bushy-directory` branch which introduces a new blob directory structure to avoid scalability issues with filesystems like ext2/3 that only accept limited numbers of entries in a directory. This branch is being used in production but needs a little which should happen in the next days. (The packing is reported to block.) At last there's a small `ctheune-betterimport` branch which fixes ZEXP import for weakref objects and has been sitting around for almost a year now too. Christian ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
Jim Fulton wrote: On Aug 1, 2008, at 11:13 AM, Wichert Akkerman wrote: Jim Fulton wrote: On Jul 31, 2008, at 1:53 PM, Chris Withers wrote: What I'd *really* like is a stable zodb release with Christian's patches for zeoraid and Shane's patches for RelStorage that then feeds through into a stable release of Zope 2. I'm not familiar with Christians patches. If they are bug fixes, they could make it into 3.8. I plan to include Shane's patches in 3.9. I'm not sure when there will be a stable release of 3.9. Do you know where there will be a new 3.8 stable release? 3.8 is stable now. We could make a 3.8.1 any time. I think there are probably going to need to be fixes to blob packing code, which could go into 3.8.1 or 3.8.2, depending on timing. Did you mean 3.9? If so, I have no idea. It's not a very high priority to me. I meant 3.8. I'ld like to use both the ZEO authentication fix and your caching fixes. Wichert. -- Wichert Akkerman[EMAIL PROTECTED]It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Aug 1, 2008, at 11:13 AM, Wichert Akkerman wrote: Jim Fulton wrote: On Jul 31, 2008, at 1:53 PM, Chris Withers wrote: What I'd *really* like is a stable zodb release with Christian's patches for zeoraid and Shane's patches for RelStorage that then feeds through into a stable release of Zope 2. I'm not familiar with Christians patches. If they are bug fixes, they could make it into 3.8. I plan to include Shane's patches in 3.9. I'm not sure when there will be a stable release of 3.9. Do you know where there will be a new 3.8 stable release? 3.8 is stable now. We could make a 3.8.1 any time. I think there are probably going to need to be fixes to blob packing code, which could go into 3.8.1 or 3.8.2, depending on timing. Did you mean 3.9? If so, I have no idea. It's not a very high priority to me. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Jul 31, 2008, at 17:23 , Chris Withers wrote: I didn't realise (and I've done it often and never experienced this in the past) than running a zopectl run script from the same instance home as a running client could result in ZEO client cache corruption. Should this be the case? Sounds logical to me. The running instance and your zopectl run-script work in the same space with identical configurations, so both will start using the same ZEO cache file. To me this sounds like a recipe for disaster. You should run your script in a dedicated instance home used for that purpose. jens ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
Jens Vagelpohl wrote: Sounds logical to me. The running instance and your zopectl run-script work in the same space with identical configurations, so both will start using the same ZEO cache file. To me this sounds like a recipe for disaster. You should run your script in a dedicated instance home used for that purpose. Hmmm, like I said, I do this often and have never had problems before. However, I suspect the difference-that-makes-a-difference is the persistent cache files. With non-persistent cache files, iirc, the files are stored in /tmp as temp files under unique names, so this kind of problem can't happen. Can anyone confirm my guesswork? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Jul 31, 2008, at 17:37 , Chris Withers wrote: Jens Vagelpohl wrote: Sounds logical to me. The running instance and your zopectl run- script work in the same space with identical configurations, so both will start using the same ZEO cache file. To me this sounds like a recipe for disaster. You should run your script in a dedicated instance home used for that purpose. Hmmm, like I said, I do this often and have never had problems before. Never had problems before doesn't make this good practice. jens ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Thu, Jul 31, 2008 at 11:37 AM, Chris Withers [EMAIL PROTECTED] wrote: However, I suspect the difference-that-makes-a-difference is the persistent cache files. With non-persistent cache files, iirc, the files are stored in /tmp as temp files under unique names, so this kind of problem can't happen. As Jens noted, this is because you're sharing a cache file. That's one of those don't do that things. It's often useful to have a separate instance that's used for run scripts and debugging; configuring that not to use a persistent cache is easy, and essential if you want to be able to run any number of them at a time. -Fred -- Fred L. Drake, Jr. fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
Chris, Are you seeing this error? https://bugs.launchpad.net/zodb/+bug/114814 If so maybe Jim's latest changes fixed this in ZODB3.8.1b65 -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
Alan Runyan wrote: Chris, Are you seeing this error? Nope, don't think so. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Jul 31, 2008, at 1:53 PM, Chris Withers wrote: Jim Fulton wrote: The most recent releases of ZODB 3.8 have *numerous* cache- management fixes. They also lock persistent cache files so you can't corrupt them by trying to open them in multiple processes. Some of the fixes affect non-persistent as well as persistent cache files. Has zodb 3.8 made it into a stable zope 2 release? I doubt it. These are all 3.8.1 beta releases. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
--On 31. Juli 2008 18:53:18 +0100 Chris Withers [EMAIL PROTECTED] wrote: Jim Fulton wrote: The most recent releases of ZODB 3.8 have *numerous* cache-management fixes. They also lock persistent cache files so you can't corrupt them by trying to open them in multiple processes. Some of the fixes affect non-persistent as well as persistent cache files. Has zodb 3.8 made it into a stable zope 2 release? Zope 2.11 ships with ZODB 3.8.0. What I'd *really* like is a stable zodb release with Christian's patches for zeoraid and Shane's patches for RelStorage that then feeds through into a stable release of Zope 2. They aren't part of the ZODB 3.8 but part of the trunk/3.9 - right? What needs to happen for that to happen? Since those changes/patches won't show in a ZODB 3.8, you likely have to wait for ZODB 3.9 which would appear in Zope 2.12. Everything else would be against our basic release policies. An eggified Zope 2 version (perhaps with 2.12) should allows to exchange the ZODB versions more easily *sigh* Andreas pgpTyN3DYz8Lm.pgp Description: PGP signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
On Thu, Jul 31, 2008 at 2:15 PM, Andreas Jung [EMAIL PROTECTED] wrote: They aren't part of the ZODB 3.8 but part of the trunk/3.9 - right? These are bug fixes, and will be included in ZODB 3.8.1. -Fred -- Fred L. Drake, Jr. fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] what could cause this kind of cache corruption?
--On 31. Juli 2008 14:53:05 -0400 Fred Drake [EMAIL PROTECTED] wrote: On Thu, Jul 31, 2008 at 2:15 PM, Andreas Jung [EMAIL PROTECTED] wrote: They aren't part of the ZODB 3.8 but part of the trunk/3.9 - right? These are bug fixes, and will be included in ZODB 3.8.1. So we will see those changes in the next Zope 2.11 release. Andreas pgpSEyfo5lU7A.pgp Description: PGP signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
