[Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
Hey, I'm in front of the same wall that Fred was going up. ;) I talked to Zagy about the problems we're facing with the current egg releases of projects that live in Zope3/src/zope/ and Zope3/src/zope/app/ and wrote this up a bit: Problems When working with eggs, we need to carefully use version numbers for the releases and be able to do continuous releases. This is currently a bit icky: - Coding in the tree doesn't force you to get the individual dependencies right, especially when dependencies evolve over time and require constraints based on the version number. - Releasing and tagging doesn't go together very well in this schema, although continuous releases shouldn't require tags anyway. The normal continuous releases from setuptools get somewhat awkward because we have the indirection of the external. I was trying to think about the constraints of handling the large tree when moving the code of all projects into satellite projects. IIRC the requirement is that all projects that we move out now will get a common version number. Proposal Here is what we found would be workable and would like to do later today before doing the beta release. I call this approach synchronized satellites picking up on the terminology Fred came up with. ;) - Move the code of the remaining projects into the satellite projects. - Replace the satellite's external with the actual code on its trunk and point the Zope 3 trunk back to the package in the satellite project's trunk. - Create a directory version in the Zope 3 tree that holds a version.txt file. This will be the common version number that Zope 3 and the satellite projects share. - Create an external in the satellite projects trunk that points to the version directory in Zope 3's trunk. - Set the version.txt file to read 3.4.0b1. This will always read the next version that is going to be released. - Change the setup.py for the satellite projects to read their version number from version/version.txt In this setting we can develop the satellite projects on their own and make releases that match the code. Future releases When releasing Zope 3 the large project as beta 1 in this case, following steps would be involved to keep the projects in sync: - Create a release tag on the satellites (tags/3.4.0b1) - Create a release tag on the Zope 3 trunk (tags/3.4.0b1) - Update the `version` external on the tagged satellites to the tag of the Zope 3 trunk - Update the Zope 3 tag to refer to the release tags on the satellites. - Increase the version number on the Zope 3 trunk to `3.4.0b2` Note: When deciding that we don't target b2 anymore but c1, we can just update the trunk at any point in time. I don't expect any hassles from that change. Most important thing is that the version.txt on the trunk is monotonically increasing. Support scripts --- As we are dealing with more than 90 eggs here, we'll also need script support to keep the handling of the synchronized satellites bearable. I'll expect to need following scripts: - Transform the current source tree into the synchronized satellites approach (one time thing) - Do a release tag of the Zope 3 tree that includes the mechanics of updating the externals as described in the section 'Future releases' (needed in the future) For those scripts I'll need to maintain a list of those synchronized satellite projects that needs to live somewhere. Not all zope.* and zope.app.* projects are satellites, just because they are referenced as externals from the Zope 3 tree (e.g. zope.testing is not) so I'm afraid I'll need to put an explicit list somewhere. Conclusion -- I hope the proposed approach sounds reasonable to you as well and would like to get feedback. I'd like to keep the beta on hold until we solved the packaging issues that I (and others) are facing currently and am willing to put in work today to keep the schedule if we can agree on a schema how to handle this. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: In a small side note: I still don't know how we recommend using the eggs in a way that says I want the egg-version of Zope 3.4.0 and the users gets those eggs (and only those) that are synchronised satellites and have 3.4.0 as their version number. I don't think we've defined what the egg-version of Zope 3.4.0 means (nor do I think we need to as part of a classic zpkg-based Zope 3.4.0). The 'and have 3.4.0 as their version number' I consider a non-goal. Maybe a meta-egg that would introduce the dependency of zope.xx==3.4.0 as their dependency would allow that to happen because if this gets involved in a working set then the synchronized satellites would be restricted to the exact version. Are you suggesting the satellites would have this dependency? If so, -1. If not, and this is just used as a way of collecting the eggs, I think this isn't valuable, and hence is a decoy. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
Hey, Am Donnerstag, den 03.05.2007, 08:59 -0400 schrieb Fred Drake: On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: In a small side note: I still don't know how we recommend using the eggs in a way that says I want the egg-version of Zope 3.4.0 and the users gets those eggs (and only those) that are synchronised satellites and have 3.4.0 as their version number. I don't think we've defined what the egg-version of Zope 3.4.0 means (nor do I think we need to as part of a classic zpkg-based Zope 3.4.0). The 'and have 3.4.0 as their version number' I consider a non-goal. Maybe a meta-egg that would introduce the dependency of zope.xx==3.4.0 as their dependency would allow that to happen because if this gets involved in a working set then the synchronized satellites would be restricted to the exact version. Are you suggesting the satellites would have this dependency? If so, -1. If not, and this is just used as a way of collecting the eggs, I think this isn't valuable, and hence is a decoy. I might have stated my goals the wrong way. I find it valuable to be able to predict which exact versions of things get pulled in from a buildout. The current way that dependencies are declared is that when I run buildout in a year I'll get the zope.app.publication-3.5dev-r122 egg on a stable application. I don't want that. I want to again and again and again get the 3.4 egg of zope.app.publication because those eggs where tested together. OTOH. I'm just remembering that Jim talked about some `freeze` feature for buildout ... is that what I seem to want to tackle this issue? ;) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
Am Donnerstag, den 03.05.2007, 08:55 -0400 schrieb Fred Drake: On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: I'm in front of the same wall that Fred was going up. ;) And I ran up against a related issue again on Tuesday, which I've not had the time to write up. - Move the code of the remaining projects into the satellite projects. - Replace the satellite's external with the actual code on its trunk and point the Zope 3 trunk back to the package in the satellite project's trunk. +1 -- The sooner, the better. - Create a directory version in the Zope 3 tree that holds a version.txt file. This will be the common version number that Zope 3 and the satellite projects share. Once the code is in the satellites, there's no reason for them to share version numbers. Ever. They can all start off with 3.4b1, and go from there. The goal is to separate the release cycles. I'm fine with that being complete for 3.4 final, but I certainly don't want the satellite projects to be tied to the Zope 3 trunk at all. - Create an external in the satellite projects trunk that points to the version directory in Zope 3's trunk. -1 -- The dependency relationship should be one-way. - Set the version.txt file to read 3.4.0b1. This will always read the next version that is going to be released. - Change the setup.py for the satellite projects to read their version number from version/version.txt -1 -- Just the version number for the satellite, please! In this setting we can develop the satellite projects on their own and make releases that match the code. Tying their version numbers to Zope 3 doesn't affect this. I'm building up on a comment by Jim who wanted to keep those synchronized. If we don't want that then the effort for keeping them synchronized would go away of course. Many things in this proposal are an effect of that requirement. Future releases When releasing Zope 3 the large project as beta 1 in this case, following steps would be involved to keep the projects in sync: - Create a release tag on the satellites (tags/3.4.0b1) -1 The branched/tagged Zope 3 should refer to specific tags/branches/revisions of the satellites, but the satellite projects should not be affected by Zope 3 releases. I'll expect to need following scripts: ... - Do a release tag of the Zope 3 tree that includes the mechanics of updating the externals as described in the section 'Future releases' (needed in the future) This should not be needed, because the satellite projects should not be affected by subsequent Zope 3 releases. Same as above. ;) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: I might have stated my goals the wrong way. I find it valuable to be able to predict which exact versions of things get pulled in from a buildout. Me too; that's very, very, very important to me. The current way that dependencies are declared is that when I run buildout in a year I'll get the zope.app.publication-3.5dev-r122 egg on a stable application. I don't want that. I want to again and again and again get the 3.4 egg of zope.app.publication because those eggs where tested together. OTOH. I'm just remembering that Jim talked about some `freeze` feature for buildout ... is that what I seem to want to tackle this issue? ;) Jim's already implemented a versions feature in zc.buildout, and I find it immensely helpful. You can set buildout:versions to the name of a section that contains version requirements. Each of those requirements is for an exact version. For example: -- [buildout] versions = versions [versions] feedparser = 4.1 -- When I build this, I get feedparser 4.1 exactly; no other version will show up. I generally nail down every version in my buildouts (including buildout recipes), except for software that I'm specifically choosing to track development for (usually application-related), and for deployments, I nail those down as well. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com