Re: Managing Debian packages with Subversion
#include hallo.h * Jamin W. Collins [Fri, Aug 29 2003, 03:13:44PM]: What _stupidness_ are you refering to? I've always just used something like: svn merge http://pkg/upstream/a.b http://pkg/upstream/current trunk Always? Nice to see that it can work with three arguments and apply the changes to the working direcotry. That is something _not_ documented in the book and is easy to be overseen in the --help message. Where http://pkg/upstream/current always contains the latest version of the upstream source and http://pkg/upstream/a.b is a tag of it (svn cp) at some time in the past. I maintain http://pkg/upstream/current with svn_load_dirs. Another example of bad documented crap, without useful help messages... svn_load_dirs file:///home/user/rep-svn/svn-testing/lilo-x lilo-22.2 lilo-22.5.7.2 /usr/bin/svn_load_dirs: import_dir `lilo-22.2' is a directory. AARGH, of course it is one, I want to specify it as the current upstream dir. Enough for today, I am kinda pissed after having to deal with the way of managing svn. I think I should write all the needed commands together in the survival-guide-for-new-svn-users. MfG, Eduard. -- Solange es Haare gibt, liegen sich die Menschen in denselben. -- Heinz Erhardt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
Eduard Bloch wrote: However, a possible implementation failed completely because of stupideness of svn merge. So we have to find an alternative solution with diffpatchcp. OTOH it is not possible to keep the version history of every file. I think you're looking for svn_load_dirs. -- see shy jo pgp0.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
On Sat, Aug 30, 2003 at 12:33:29AM +0200, Eduard Bloch wrote: #include hallo.h * Jamin W. Collins [Fri, Aug 29 2003, 03:13:44PM]: What _stupidness_ are you refering to? I've always just used something like: svn merge http://pkg/upstream/a.b http://pkg/upstream/current trunk Always? Nice to see that it can work with three arguments and apply the changes to the working direcotry. That is something _not_ documented in the book and is easy to be overseen in the --help message. It is documented in the book, that's where I learned how to use it. http://svnbook.red-bean.com/html-chunk/ch06s04.html#svn-ch-6-sect-4.1 svn merge http://svn.example.com/repos/vendor/libcomplex/1.0 \ http://svn.example.com/repos/vendor/libcomplex/current \ libcomplex Where http://pkg/upstream/current always contains the latest version of the upstream source and http://pkg/upstream/a.b is a tag of it (svn cp) at some time in the past. I maintain http://pkg/upstream/current with svn_load_dirs. Another example of bad documented crap, without useful help messages... svn_load_dirs file:///home/user/rep-svn/svn-testing/lilo-x lilo-22.2 lilo-22.5.7.2 /usr/bin/svn_load_dirs: import_dir `lilo-22.2' is a directory. The second command line option must be a path relative to the first in the repository, not your working copy or a new directory of upstream sources. The third command line option is the first of the new upstream source directories. Again, documented fairly clearly in the book: svn_load_dirs.pl takes three mandatory arguments. The first argument is the URL to the base Subversion directory to work in. This argument is followed by the URL--relative the first argument--into which the current vendor drop will be imported. Finally, the third argument is the local directory to import. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
On Thu, Aug 28, 2003 at 09:56:29PM -0400, Joey Hess wrote: Jamin W. Collins wrote: I prefer an alternate structure where you start with the project as a top level. This keeps all project (package) related files under one directory in the project: project/ trunk/ branches/ tags/ vendor/ The only problem with this is that if you maintain N packages, and you presumably set up svn:externals pulling in all N, so you can update your whole source tree with one svn up Learn something new every day. I've never known about svn:externals. I can see how this could cause problems. However, the above structure still seems more organized to me. Either way, I suggest the addition of a vendor/ directory to track differences in upstream versions. The reason for this will be clear (see below). Vendor is just a branch so I don't understand why it should be kept separate. Just a matter of organization I guess. I just like keeping the vendor branches distinctly different from the package release branches. This way I know anything under project/vendor/ is an upstream source version and anything under project/tag/ is a tagged pacakge release. So, both work, just a matter of what works for you conceptually and how you use it. -- Jamin W. Collins -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
On Fri, Aug 29, 2003 at 12:44:02PM +0300, Fabian Fagerholm wrote: * Check out the trunk you just created into a directory called project-version (svn co). I was with you up to this point. I don't normally name my working copy in the form project-version as that form is needed for actually making hte package and for that we need an exported copy of the working copy. Since I use layout A.A, my working copy is normally simply named trunk. All work is done in it. My initial dh_make is run in an export of this named project-version and then project-version/debian directory is copied to trunk/debian and added to Subversion's control (svn add). Not sure if this is the easiest way to go about the initial steps, just the way I've been working with it. and the new upstream version. (This section is largely based on the Subversion Book) * Download the new upstream source and unpack it. * Use the svn_load_dirs tool to update the current upstream version and tag it (this will be committed immediately): * A.A. svn_load_dirs -t version url://host/repos/projectX/vendor current /path/to/new/version Might want to note that the version in these should probably reflect upstreams version naming/number. versions. (You should do so in any case...) * Check out HEAD (the latest version) of your trunk (svn co). * Merge the differences between the previous upstream version and the current (new) upstream version into your trunk (svn merge): svn merge url://to/previous/version url://to/current/version trunk Again, might want to indicate that the url://to/previous/version and url://to/current/version are referencing the upstream branch or vendor tags. -- Jamin W. Collins To be nobody but yourself when the whole world is trying it's best night and day to make you everybody else is to fight the hardest battle any human being will fight. -- E.E. Cummings -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
#include hallo.h * Fabian Fagerholm [Fri, Aug 29 2003, 12:44:02PM]: On Thu, 2003-08-28 at 22:24, Fabian Fagerholm wrote: How do I manage a new Debian package with Subversion? Thanks to [EMAIL PROTECTED] and [EMAIL PROTECTED] for their insight into this issue. I will try to summarize everything in this reply to myself. Managing Debian packages with Subversion The following assumes you have basic knowledge of Subversion (SVN). If you need to read up on its installation and usage, please consult the Subversion Book by Ben Collins-Sussman, Brian W. Fitzpatrick, C. Michael Pilato, available from the Subversion site at http://subversion.tigris.org/ or the book's site http://svnbook.red-bean.com/. What are we trying to accomplish? We want to manage Debian packages with Subversion. In other words, we want to store all the files related to a Debian source package in a Subversion repository, including the upstream source. When making changes to the packaging, or when updating the package to a new upstream version, we want to check out our project (our package) from the repository, make the required changes, and then commit the changes back into the repository. Also, when we are satisfied with the state of the packaging and want to release a new version, we want to automate certain steps such as tagging the current state of the repository with an appropriate name and version number (corresponding to the debian package version), exporting this tag and building the package. Where appropriate, we want to use the Subversion way of doing these things, meaning we want to take full advantage of the features provided by Subversion. Choosing your repository structure While there are several ways of structuring your repository, common practise is to have trunk/, branches/ and tags/ directories and divide the content among these. Two questions arise: * Should the repository have trunk/, branches/ and tags/ at the top level, each containing project directories -- or should the top level be divided by project, and each project directory contain trunk/, branches/ and tags/? * Should there be an additional top-level directory called vendor/, containing the upstream source -- or should the upstream source be treated as just another branch? We thus have 2x2 choices: A. Use vendor/ for upstream source. A. Per-project trunk/, branches/, tags/ and vendor/. B. Repository-wide trunk/, branches/, tags/ and vendor/. B. Use a regular branch for upstream source. A. Per-project trunk/, branches/ and tags. B. Repository-wide trunk/, branches/ and tags/. Choosing one is largely a matter of taste. The repository will look clean and structured to newcomers if the top-level directory is divided by project (sub-option A). It might also add to ease of comprehension to have upstream sources in vendor/ (option A). But there are some technical considerations if you want to use the svn:externals property (see the Subversion Book for details) to pull in several project directories at once. (TODO: explain this further) Option A.A.: Per-project trunk/, branches/, tags/ and vendor/ The repository will have the following structure: projectA/ trunk/ branches/ tags/ vendor/ projectB/ trunk/ branches/ tags/ vendor/ ... The idea of this layout is to reflect the logical structure in the repository. Internally, Subversion makes no distinction between a directory called trunk and a directory called tags. Only your use of these directories dictate their meaning. To Subversion, it's all the same. Option A.B.: Repository-wide trunk/, branches/, tags/ and vendor/ The repository will have the following structure: trunk/ projectA/ projectB/ ... branches/ projectA/ projectB/ ... tags/ projectA/ projectB/ ... vendor/ projectA/ projectB/ ... The advantage of this might be that the svn:externals property might work better. (TODO: verify, why) Option B.A.: Per-project trunk/, branches/ and tags/ The repository will have the following structure: projectA/ trunk/ branches/ tags/ projectB/ trunk/ branches/ tags/ ... Here, the logical structure is reflected in the repository, but the upstream sources are considered just another branch. Vendor tags are considered just another set of tags. This approach is fine, but there is no technical reason to do so. Subversion doesn't care. Option B.B.: Repository-wide trunk/, branches/ and tags
Re: Managing Debian packages with Subversion
Fabian Fagerholm wrote: Thanks to [EMAIL PROTECTED] and [EMAIL PROTECTED] for their insight into this issue. I will try to summarize everything in this reply to myself. Nice job writing this up, here are some corrections. The repository will have the following structure: trunk/ projectA/ projectB/ ... branches/ projectA/ projectB/ ... tags/ projectA/ projectB/ ... vendor/ projectA/ projectB/ ... The advantage of this might be that the svn:externals property might work better. (TODO: verify, why) The reason I prefer this layout is I can check out the whole trunk/ directory here, and get a nice directory with all my debian packages (projectA, projectB, projectC..) in it. No need for svn:externals at all. Option B.A.: Per-project trunk/, branches/ and tags/ The repository will have the following structure: projectA/ trunk/ branches/ tags/ projectB/ trunk/ branches/ tags/ ... Compare with a layout like this one. If I want a directory with all my projects, I must either check out the whole repository (likely not an option due to disk spack), or set up a directory with a svn:externals propery like this: projectAsvn://wherever/projectA projectBsvn://wherever/projectB projectCsvn://wherever/projectC And maintain this. The maintenance burden, coupled with the limitations of svn:externals (which some subversion developers don't like and would prefer to just remove), make me prefer to avoid svn:externals when possible. Of course, there are advantages to the toplevel project layouts too. For one thing, the tags and branches directories are closer to the trunk directory, which can make it be less typing to tag projects, work with branches, and so on. Just do a svn info, and paste the repo url, changing the last word of it from trunk to tags. I've lately been working on tools to automate this kind of thing, for example I have a svnpath tool that I use to get directories for tags and branches. It works with either repo layout, like this: svn cp `svnpath` `svnpath tags`/new-tag Will make a new tag of the project I'm cd'd to. This is in my subversion repository in the bin directory along with other tools to do automated committing and tagging based on the debian/changelog. * B.B. svn_load_dirs -t tags/projectX/upstream/version url://host/repos/ branches/projectX/upstream /path/to/new/version I've only ever had to do this once, and I tried a svn_load_dirs command like that and watched it try to check out the root of my repository. 10+ gb of files (hundreds of versions of most packages) over dialup. The thing is very badly documented, but I eventually worked out that I could run it like this: svn_load_dirs url://host/repos/branches/projectX upstream /dir This leaves me doing the tag myself of course. Even when run this way it apparently checked out all of url://host/repos/branches/projectX/, which could be quite painful if you have a lot of branches. I hope this tool gets improved. Then export (svn export) the current tag and build the package. You also need to export the upstream source and turn it into an .orig.tar.gz. Please don't do that. Keep an archive of your .orig.tar.gz's and copy the tarball into place before doing the build so we can have pristine source. I personally use a local mini-dinstall repository that I upload all my packages too, that way I have all the released packages handy, along with their sources. Another approach is the check the orig tarballs into svn, if you're optimistic about the ever-falling price of disk space. :-) -- see shy jo pgp0.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
Jamin W. Collins wrote: I was with you up to this point. I don't normally name my working copy in the form project-version as that form is needed for actually making hte package and for that we need an exported copy of the working copy. Actually, dpkg-buildpackage does not care what you name the build directory of a package. Persaonally, I use just packagename for the directory names and build directly from them, using a collection of ugly hacks to ignore the .svn directories[1]. I don't really recommend that, but it works. -- see shy jo [1] DH_ALWAYS_EXCLUDE=CVS:.svn dpkg-buildpackage -i'(?:^|/).*~$|(?:^|/)\..*\.swp|DEADJOE|(?:/CVS|/.svn|/RCS|/.deps)(?:$|/.*$)' -ICVS -I.svn pgp0.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
On Fri, Aug 29, 2003 at 12:30:46PM -0400, Joey Hess wrote: Fabian Fagerholm wrote: * B.B. svn_load_dirs -t tags/projectX/upstream/version url://host/repos/ branches/projectX/upstream /path/to/new/version I've only ever had to do this once, and I tried a svn_load_dirs command like that and watched it try to check out the root of my repository. 10+ gb of files (hundreds of versions of most packages) over dialup. The thing is very badly documented, but I eventually worked out that I could run it like this: svn_load_dirs url://host/repos/branches/projectX upstream /dir This leaves me doing the tag myself of course. Even when run this way it apparently checked out all of url://host/repos/branches/projectX/, which could be quite painful if you have a lot of branches. I hope this tool gets improved. I believe they corrected this behaviour in recent versions. A few others noticed the fact that svn_load_dirs would check out everything below the first parameter in the repository too. There was some discussion about this on their dev list: http://www.contactor.se/~dast/svn/archive-2003-07/2043.shtml The issue opened for it was: http://subversion.tigris.org/issues/show_bug.cgi?id=1455 Which appears to have been included in the 0.27 release. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
#include hallo.h * Joey Hess [Fri, Aug 29 2003, 12:35:46PM]: Actually, dpkg-buildpackage does not care what you name the build directory of a package. Persaonally, I use just packagename for the directory names and build directly from them, using a collection of ugly hacks to ignore the .svn directories[1]. I don't really recommend that, but it works. Question: how do you manage upstream and Debian branches? I encounter this problem while trying to find the best concept for svn-uupdate. The first idea was: branches/upstream/a.b --- branches/upstream/a.c | (svn cp + copy of new src) | | (svn cp + .diff.gz) | | | (svn merge w/ trunk) V V trunk -updated trunk | (svn merge with a.c) | V tags/a.b-7 However, a possible implementation failed completely because of stupideness of svn merge. So we have to find an alternative solution with diffpatchcp. OTOH it is not possible to keep the version history of every file. Upstream versions should be tracked with svn (svn copy x y ; cd y ; svn status | grep ^? | xargs svn add), keeping the history of the upstream files and saving the space in the repository. But how to proceed with changes in the files and debian directory itself? Simplest solution: copy each upstream version a.b to a.b-c and apply the Debian patch from the previous version. Pro: easy to be implemented. Contra: loosing version history of the particular file(s). Next idea: follow this pseudocode: foreach(files modified in debian-diff) ; if (file doesn't exist in a.c-1 yet) svn cp file from a.b-7 to a.c-1 else apply the diff part for this file on a.c-1 fi done Following this principle, we would preserve as much history data as possible. Changes on upstream files are not so good visible then, but it is better than pure patchsvn add approach. Any ideas? I feel like to implement the last thing. MfG, Eduard. -- Hallo Gartenzwergsammler! pgp0.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
On Fri, Aug 29, 2003 at 10:30:46PM +0200, Eduard Bloch wrote: Question: how do you manage upstream and Debian branches? I encounter this problem while trying to find the best concept for svn-uupdate. The first idea was: branches/upstream/a.b --- branches/upstream/a.c | (svn cp + copy of new src) | | (svn cp + .diff.gz) | | | (svn merge w/ trunk) V V trunk -updated trunk | (svn merge with a.c) | V tags/a.b-7 However, a possible implementation failed completely because of stupideness of svn merge. What _stupidness_ are you refering to? I've always just used something like: svn merge http://pkg/upstream/a.b http://pkg/upstream/current trunk Where http://pkg/upstream/current always contains the latest version of the upstream source and http://pkg/upstream/a.b is a tag of it (svn cp) at some time in the past. I maintain http://pkg/upstream/current with svn_load_dirs. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
On Fri, Aug 29, 2003 at 12:44:02PM +0300, Fabian Fagerholm wrote: * Check out the trunk you just created into a directory called project-version (svn co). I was with you up to this point. I don't normally name my working copy in the form project-version as that form is needed for actually making hte package and for that we need an exported copy of the working copy. Since I use layout A.A, my working copy is normally simply named trunk. All work is done in it. My initial dh_make is run in an export of this named project-version and then project-version/debian directory is copied to trunk/debian and added to Subversion's control (svn add). Not sure if this is the easiest way to go about the initial steps, just the way I've been working with it. and the new upstream version. (This section is largely based on the Subversion Book) * Download the new upstream source and unpack it. * Use the svn_load_dirs tool to update the current upstream version and tag it (this will be committed immediately): * A.A. svn_load_dirs -t version url://host/repos/projectX/vendor current /path/to/new/version Might want to note that the version in these should probably reflect upstreams version naming/number. versions. (You should do so in any case...) * Check out HEAD (the latest version) of your trunk (svn co). * Merge the differences between the previous upstream version and the current (new) upstream version into your trunk (svn merge): svn merge url://to/previous/version url://to/current/version trunk Again, might want to indicate that the url://to/previous/version and url://to/current/version are referencing the upstream branch or vendor tags. -- Jamin W. Collins To be nobody but yourself when the whole world is trying it's best night and day to make you everybody else is to fight the hardest battle any human being will fight. -- E.E. Cummings
Re: Managing Debian packages with Subversion
Jamin W. Collins wrote: I prefer an alternate structure where you start with the project as a top level. This keeps all project (package) related files under one directory in the project: project/ trunk/ branches/ tags/ vendor/ The only problem with this is that if you maintain N packages, and you presumably set up svn:externals pulling in all N, so you can update your whole source tree with one svn up, then subversion will currently open n connections to your server. Due to some bugs it will keep all N connections open until it's done with the whole update. If you use subversion over ssh, this is pretty bad. I have successfully fork bombed my server with subversion. (N was 40 or so.) Not to mention all the other limitations of svn:externals. It's fine if you're only maintaining a few packages I guess. Either way, I suggest the addition of a vendor/ directory to track differences in upstream versions. The reason for this will be clear (see below). Vendor is just a branch so I don't understand why it should be kept separate. I use branches/package/upstream for this, and put vendor tags in tags/package/upstream_revision_x.y. I am considering changing the latter, which is derived from what cvs2svn does to old cvs-buildpackage managed repositories, to something like tags/package/upstream/x.y -- see shy jo pgpLtPInbASYC.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
On Thu, Aug 28, 2003 at 09:56:29PM -0400, Joey Hess wrote: Jamin W. Collins wrote: I prefer an alternate structure where you start with the project as a top level. This keeps all project (package) related files under one directory in the project: project/ trunk/ branches/ tags/ vendor/ The only problem with this is that if you maintain N packages, and you presumably set up svn:externals pulling in all N, so you can update your whole source tree with one svn up Learn something new every day. I've never known about svn:externals. I can see how this could cause problems. However, the above structure still seems more organized to me. Either way, I suggest the addition of a vendor/ directory to track differences in upstream versions. The reason for this will be clear (see below). Vendor is just a branch so I don't understand why it should be kept separate. Just a matter of organization I guess. I just like keeping the vendor branches distinctly different from the package release branches. This way I know anything under project/vendor/ is an upstream source version and anything under project/tag/ is a tagged pacakge release. So, both work, just a matter of what works for you conceptually and how you use it. -- Jamin W. Collins
Re: Managing Debian packages with Subversion
#include hallo.h * Fabian Fagerholm [Fri, Aug 29 2003, 12:44:02PM]: On Thu, 2003-08-28 at 22:24, Fabian Fagerholm wrote: How do I manage a new Debian package with Subversion? Thanks to [EMAIL PROTECTED] and [EMAIL PROTECTED] for their insight into this issue. I will try to summarize everything in this reply to myself. Managing Debian packages with Subversion The following assumes you have basic knowledge of Subversion (SVN). If you need to read up on its installation and usage, please consult the Subversion Book by Ben Collins-Sussman, Brian W. Fitzpatrick, C. Michael Pilato, available from the Subversion site at http://subversion.tigris.org/ or the book's site http://svnbook.red-bean.com/. What are we trying to accomplish? We want to manage Debian packages with Subversion. In other words, we want to store all the files related to a Debian source package in a Subversion repository, including the upstream source. When making changes to the packaging, or when updating the package to a new upstream version, we want to check out our project (our package) from the repository, make the required changes, and then commit the changes back into the repository. Also, when we are satisfied with the state of the packaging and want to release a new version, we want to automate certain steps such as tagging the current state of the repository with an appropriate name and version number (corresponding to the debian package version), exporting this tag and building the package. Where appropriate, we want to use the Subversion way of doing these things, meaning we want to take full advantage of the features provided by Subversion. Choosing your repository structure While there are several ways of structuring your repository, common practise is to have trunk/, branches/ and tags/ directories and divide the content among these. Two questions arise: * Should the repository have trunk/, branches/ and tags/ at the top level, each containing project directories -- or should the top level be divided by project, and each project directory contain trunk/, branches/ and tags/? * Should there be an additional top-level directory called vendor/, containing the upstream source -- or should the upstream source be treated as just another branch? We thus have 2x2 choices: A. Use vendor/ for upstream source. A. Per-project trunk/, branches/, tags/ and vendor/. B. Repository-wide trunk/, branches/, tags/ and vendor/. B. Use a regular branch for upstream source. A. Per-project trunk/, branches/ and tags. B. Repository-wide trunk/, branches/ and tags/. Choosing one is largely a matter of taste. The repository will look clean and structured to newcomers if the top-level directory is divided by project (sub-option A). It might also add to ease of comprehension to have upstream sources in vendor/ (option A). But there are some technical considerations if you want to use the svn:externals property (see the Subversion Book for details) to pull in several project directories at once. (TODO: explain this further) Option A.A.: Per-project trunk/, branches/, tags/ and vendor/ The repository will have the following structure: projectA/ trunk/ branches/ tags/ vendor/ projectB/ trunk/ branches/ tags/ vendor/ ... The idea of this layout is to reflect the logical structure in the repository. Internally, Subversion makes no distinction between a directory called trunk and a directory called tags. Only your use of these directories dictate their meaning. To Subversion, it's all the same. Option A.B.: Repository-wide trunk/, branches/, tags/ and vendor/ The repository will have the following structure: trunk/ projectA/ projectB/ ... branches/ projectA/ projectB/ ... tags/ projectA/ projectB/ ... vendor/ projectA/ projectB/ ... The advantage of this might be that the svn:externals property might work better. (TODO: verify, why) Option B.A.: Per-project trunk/, branches/ and tags/ The repository will have the following structure: projectA/ trunk/ branches/ tags/ projectB/ trunk/ branches/ tags/ ... Here, the logical structure is reflected in the repository, but the upstream sources are considered just another branch. Vendor tags are considered just another set of tags. This approach is fine, but there is no technical reason to do so. Subversion doesn't care. Option B.B.: Repository-wide trunk/, branches/ and tags
Re: Managing Debian packages with Subversion
Fabian Fagerholm wrote: Thanks to [EMAIL PROTECTED] and [EMAIL PROTECTED] for their insight into this issue. I will try to summarize everything in this reply to myself. Nice job writing this up, here are some corrections. The repository will have the following structure: trunk/ projectA/ projectB/ ... branches/ projectA/ projectB/ ... tags/ projectA/ projectB/ ... vendor/ projectA/ projectB/ ... The advantage of this might be that the svn:externals property might work better. (TODO: verify, why) The reason I prefer this layout is I can check out the whole trunk/ directory here, and get a nice directory with all my debian packages (projectA, projectB, projectC..) in it. No need for svn:externals at all. Option B.A.: Per-project trunk/, branches/ and tags/ The repository will have the following structure: projectA/ trunk/ branches/ tags/ projectB/ trunk/ branches/ tags/ ... Compare with a layout like this one. If I want a directory with all my projects, I must either check out the whole repository (likely not an option due to disk spack), or set up a directory with a svn:externals propery like this: projectAsvn://wherever/projectA projectBsvn://wherever/projectB projectCsvn://wherever/projectC And maintain this. The maintenance burden, coupled with the limitations of svn:externals (which some subversion developers don't like and would prefer to just remove), make me prefer to avoid svn:externals when possible. Of course, there are advantages to the toplevel project layouts too. For one thing, the tags and branches directories are closer to the trunk directory, which can make it be less typing to tag projects, work with branches, and so on. Just do a svn info, and paste the repo url, changing the last word of it from trunk to tags. I've lately been working on tools to automate this kind of thing, for example I have a svnpath tool that I use to get directories for tags and branches. It works with either repo layout, like this: svn cp `svnpath` `svnpath tags`/new-tag Will make a new tag of the project I'm cd'd to. This is in my subversion repository in the bin directory along with other tools to do automated committing and tagging based on the debian/changelog. * B.B. svn_load_dirs -t tags/projectX/upstream/version url://host/repos/ branches/projectX/upstream /path/to/new/version I've only ever had to do this once, and I tried a svn_load_dirs command like that and watched it try to check out the root of my repository. 10+ gb of files (hundreds of versions of most packages) over dialup. The thing is very badly documented, but I eventually worked out that I could run it like this: svn_load_dirs url://host/repos/branches/projectX upstream /dir This leaves me doing the tag myself of course. Even when run this way it apparently checked out all of url://host/repos/branches/projectX/, which could be quite painful if you have a lot of branches. I hope this tool gets improved. Then export (svn export) the current tag and build the package. You also need to export the upstream source and turn it into an .orig.tar.gz. Please don't do that. Keep an archive of your .orig.tar.gz's and copy the tarball into place before doing the build so we can have pristine source. I personally use a local mini-dinstall repository that I upload all my packages too, that way I have all the released packages handy, along with their sources. Another approach is the check the orig tarballs into svn, if you're optimistic about the ever-falling price of disk space. :-) -- see shy jo pgp4mysTLgNXu.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
Jamin W. Collins wrote: I was with you up to this point. I don't normally name my working copy in the form project-version as that form is needed for actually making hte package and for that we need an exported copy of the working copy. Actually, dpkg-buildpackage does not care what you name the build directory of a package. Persaonally, I use just packagename for the directory names and build directly from them, using a collection of ugly hacks to ignore the .svn directories[1]. I don't really recommend that, but it works. -- see shy jo [1] DH_ALWAYS_EXCLUDE=CVS:.svn dpkg-buildpackage -i'(?:^|/).*~$|(?:^|/)\..*\.swp|DEADJOE|(?:/CVS|/.svn|/RCS|/.deps)(?:$|/.*$)' -ICVS -I.svn pgpMNShdDGAS1.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
On Fri, Aug 29, 2003 at 12:30:46PM -0400, Joey Hess wrote: Fabian Fagerholm wrote: * B.B. svn_load_dirs -t tags/projectX/upstream/version url://host/repos/ branches/projectX/upstream /path/to/new/version I've only ever had to do this once, and I tried a svn_load_dirs command like that and watched it try to check out the root of my repository. 10+ gb of files (hundreds of versions of most packages) over dialup. The thing is very badly documented, but I eventually worked out that I could run it like this: svn_load_dirs url://host/repos/branches/projectX upstream /dir This leaves me doing the tag myself of course. Even when run this way it apparently checked out all of url://host/repos/branches/projectX/, which could be quite painful if you have a lot of branches. I hope this tool gets improved. I believe they corrected this behaviour in recent versions. A few others noticed the fact that svn_load_dirs would check out everything below the first parameter in the repository too. There was some discussion about this on their dev list: http://www.contactor.se/~dast/svn/archive-2003-07/2043.shtml The issue opened for it was: http://subversion.tigris.org/issues/show_bug.cgi?id=1455 Which appears to have been included in the 0.27 release. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar
Re: Managing Debian packages with Subversion
On Thu, Aug 28, 2003 at 10:24:57PM +0300, Fabian Fagerholm wrote: How do I manage a new Debian package with Subversion? The things I've found only scratch the surface a bit, assume a level of knowledge beyond mine, or assume you're migrating from CVS and already have made several initial decisions. Also, I'm hoping to include everything in the SVN repository, not just the debian/ directory. I'm not considering Debian native packages, but regular packages that have an upstream tarball. Here's what I have so far: * The repository will have a structure where the root contains the trunk/, branches/ and tags/ directories. I prefer an alternate structure where you start with the project as a top level. This keeps all project (package) related files under one directory in the project: project/ trunk/ branches/ tags/ vendor/ Either way, I suggest the addition of a vendor/ directory to track differences in upstream versions. The reason for this will be clear (see below). * Projects will start their life in trunk/project, branches being stored in branches/project-branchname and tags in tags/. * Initial package creation will follow the following pattern: * Download upstream source and unpack, - Import into the vendor/current folder - Tag current state of vendor/current to vendor/source-version (copy) - Populate trunk/ from vendor/current (copy) - Check out trunk/ Working with the extracted upstream: * debianize by dh_make, * integrate into SVN somehow. - copy new source-version/debian directory to trunk/debian - add the new trunk/debian directory to subversion's control - adjust packaging scripts as needed - commit changes * Development cycle will follow the following pattern: * Check out HEAD from SVN, * make the required changes, * commit. Nothing special here. Correct. * When making a release... svn export? Yes, that way you have a source structure devoid of the .svn subdirectories. I use the attached export script to ease the and build process. It assumes it is run from the project directory and that the orig source file and the svn working copy (in a directory called trunk) exist in the current directory. * When upgrading to a newer upstream version... how? This is where the vendor/ directory above helps. The Subversion book refers to this a a vendor branch. Essentially you apply a diff between the old and new upstream sources to the current devel trunk. Details on this are in the Subversion book. I stress again that I'm looking for a solution that stores *everything* in SVN, not only the debian/ directory. This is exactly what I do with the Debian packages I create and maintain. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar
Re: Managing Debian packages with Subversion
On Fri, Aug 29, 2003 at 10:30:46PM +0200, Eduard Bloch wrote: Question: how do you manage upstream and Debian branches? I encounter this problem while trying to find the best concept for svn-uupdate. The first idea was: branches/upstream/a.b --- branches/upstream/a.c | (svn cp + copy of new src) | | (svn cp + .diff.gz) | | | (svn merge w/ trunk) V V trunk -updated trunk | (svn merge with a.c) | V tags/a.b-7 However, a possible implementation failed completely because of stupideness of svn merge. What _stupidness_ are you refering to? I've always just used something like: svn merge http://pkg/upstream/a.b http://pkg/upstream/current trunk Where http://pkg/upstream/current always contains the latest version of the upstream source and http://pkg/upstream/a.b is a tag of it (svn cp) at some time in the past. I maintain http://pkg/upstream/current with svn_load_dirs. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar
Re: Managing Debian packages with Subversion
#include hallo.h * Joey Hess [Fri, Aug 29 2003, 12:35:46PM]: Actually, dpkg-buildpackage does not care what you name the build directory of a package. Persaonally, I use just packagename for the directory names and build directly from them, using a collection of ugly hacks to ignore the .svn directories[1]. I don't really recommend that, but it works. Question: how do you manage upstream and Debian branches? I encounter this problem while trying to find the best concept for svn-uupdate. The first idea was: branches/upstream/a.b --- branches/upstream/a.c | (svn cp + copy of new src) | | (svn cp + .diff.gz) | | | (svn merge w/ trunk) V V trunk -updated trunk | (svn merge with a.c) | V tags/a.b-7 However, a possible implementation failed completely because of stupideness of svn merge. So we have to find an alternative solution with diffpatchcp. OTOH it is not possible to keep the version history of every file. Upstream versions should be tracked with svn (svn copy x y ; cd y ; svn status | grep ^? | xargs svn add), keeping the history of the upstream files and saving the space in the repository. But how to proceed with changes in the files and debian directory itself? Simplest solution: copy each upstream version a.b to a.b-c and apply the Debian patch from the previous version. Pro: easy to be implemented. Contra: loosing version history of the particular file(s). Next idea: follow this pseudocode: foreach(files modified in debian-diff) ; if (file doesn't exist in a.c-1 yet) svn cp file from a.b-7 to a.c-1 else apply the diff part for this file on a.c-1 fi done Following this principle, we would preserve as much history data as possible. Changes on upstream files are not so good visible then, but it is better than pure patchsvn add approach. Any ideas? I feel like to implement the last thing. MfG, Eduard. -- Hallo Gartenzwergsammler! pgptFlf3iatKw.pgp Description: PGP signature
Re: Managing Debian packages with Subversion
On Sat, Aug 30, 2003 at 12:33:29AM +0200, Eduard Bloch wrote: #include hallo.h * Jamin W. Collins [Fri, Aug 29 2003, 03:13:44PM]: What _stupidness_ are you refering to? I've always just used something like: svn merge http://pkg/upstream/a.b http://pkg/upstream/current trunk Always? Nice to see that it can work with three arguments and apply the changes to the working direcotry. That is something _not_ documented in the book and is easy to be overseen in the --help message. It is documented in the book, that's where I learned how to use it. http://svnbook.red-bean.com/html-chunk/ch06s04.html#svn-ch-6-sect-4.1 svn merge http://svn.example.com/repos/vendor/libcomplex/1.0 \ http://svn.example.com/repos/vendor/libcomplex/current \ libcomplex Where http://pkg/upstream/current always contains the latest version of the upstream source and http://pkg/upstream/a.b is a tag of it (svn cp) at some time in the past. I maintain http://pkg/upstream/current with svn_load_dirs. Another example of bad documented crap, without useful help messages... svn_load_dirs file:///home/user/rep-svn/svn-testing/lilo-x lilo-22.2 lilo-22.5.7.2 /usr/bin/svn_load_dirs: import_dir `lilo-22.2' is a directory. The second command line option must be a path relative to the first in the repository, not your working copy or a new directory of upstream sources. The third command line option is the first of the new upstream source directories. Again, documented fairly clearly in the book: svn_load_dirs.pl takes three mandatory arguments. The first argument is the URL to the base Subversion directory to work in. This argument is followed by the URL--relative the first argument--into which the current vendor drop will be imported. Finally, the third argument is the local directory to import. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar
Re: Managing Debian packages with Subversion
#include hallo.h * Jamin W. Collins [Fri, Aug 29 2003, 03:13:44PM]: What _stupidness_ are you refering to? I've always just used something like: svn merge http://pkg/upstream/a.b http://pkg/upstream/current trunk Always? Nice to see that it can work with three arguments and apply the changes to the working direcotry. That is something _not_ documented in the book and is easy to be overseen in the --help message. Where http://pkg/upstream/current always contains the latest version of the upstream source and http://pkg/upstream/a.b is a tag of it (svn cp) at some time in the past. I maintain http://pkg/upstream/current with svn_load_dirs. Another example of bad documented crap, without useful help messages... svn_load_dirs file:///home/user/rep-svn/svn-testing/lilo-x lilo-22.2 lilo-22.5.7.2 /usr/bin/svn_load_dirs: import_dir `lilo-22.2' is a directory. AARGH, of course it is one, I want to specify it as the current upstream dir. Enough for today, I am kinda pissed after having to deal with the way of managing svn. I think I should write all the needed commands together in the survival-guide-for-new-svn-users. MfG, Eduard. -- Solange es Haare gibt, liegen sich die Menschen in denselben. -- Heinz Erhardt
Managing Debian packages with Subversion
Hi, I've read most of what is available about managing Debian packages with Subversion, but I've been unable to find a comprehensive answer to the following question: How do I manage a new Debian package with Subversion? The things I've found only scratch the surface a bit, assume a level of knowledge beyond mine, or assume you're migrating from CVS and already have made several initial decisions. Also, I'm hoping to include everything in the SVN repository, not just the debian/ directory. I'm not considering Debian native packages, but regular packages that have an upstream tarball. Here's what I have so far: * The repository will have a structure where the root contains the trunk/, branches/ and tags/ directories. * Projects will start their life in trunk/project, branches being stored in branches/project-branchname and tags in tags/. * Initial package creation will follow the following pattern: * Download upstream source and unpack, * debianize by dh_make, * integrate into SVN somehow. * Development cycle will follow the following pattern: * Check out HEAD from SVN, * make the required changes, * commit. Nothing special here. * When making a release... svn export? * When upgrading to a newer upstream version... how? I stress again that I'm looking for a solution that stores *everything* in SVN, not only the debian/ directory. Perhaps someone with greater insight could fill in the list above, and hopefully this thread will end with a nice description of how to accomplish the task. Cheers, -- Fabian Fagerholm [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: Managing Debian packages with Subversion
On Thu, Aug 28, 2003 at 10:24:57PM +0300, Fabian Fagerholm wrote: How do I manage a new Debian package with Subversion? The things I've found only scratch the surface a bit, assume a level of knowledge beyond mine, or assume you're migrating from CVS and already have made several initial decisions. Also, I'm hoping to include everything in the SVN repository, not just the debian/ directory. I'm not considering Debian native packages, but regular packages that have an upstream tarball. Here's what I have so far: * The repository will have a structure where the root contains the trunk/, branches/ and tags/ directories. I prefer an alternate structure where you start with the project as a top level. This keeps all project (package) related files under one directory in the project: project/ trunk/ branches/ tags/ vendor/ Either way, I suggest the addition of a vendor/ directory to track differences in upstream versions. The reason for this will be clear (see below). * Projects will start their life in trunk/project, branches being stored in branches/project-branchname and tags in tags/. * Initial package creation will follow the following pattern: * Download upstream source and unpack, - Import into the vendor/current folder - Tag current state of vendor/current to vendor/source-version (copy) - Populate trunk/ from vendor/current (copy) - Check out trunk/ Working with the extracted upstream: * debianize by dh_make, * integrate into SVN somehow. - copy new source-version/debian directory to trunk/debian - add the new trunk/debian directory to subversion's control - adjust packaging scripts as needed - commit changes * Development cycle will follow the following pattern: * Check out HEAD from SVN, * make the required changes, * commit. Nothing special here. Correct. * When making a release... svn export? Yes, that way you have a source structure devoid of the .svn subdirectories. I use the attached export script to ease the and build process. It assumes it is run from the project directory and that the orig source file and the svn working copy (in a directory called trunk) exist in the current directory. * When upgrading to a newer upstream version... how? This is where the vendor/ directory above helps. The Subversion book refers to this a a vendor branch. Essentially you apply a diff between the old and new upstream sources to the current devel trunk. Details on this are in the Subversion book. I stress again that I'm looking for a solution that stores *everything* in SVN, not only the debian/ directory. This is exactly what I do with the Debian packages I create and maintain. -- Jamin W. Collins This is the typical unix way of doing things: you string together lots of very specific tools to accomplish larger tasks. -- Vineet Kumar -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Managing Debian packages with Subversion
On Thu, Aug 28, 2003 at 02:13:51PM -0600, Jamin W. Collins wrote: Yes, that way you have a source structure devoid of the .svn subdirectories. I use the attached export script to ease the and build process. It assumes it is run from the project directory and that the orig source file and the svn working copy (in a directory called trunk) exist in the current directory. Guess it helps if I include the script that I referenced... -- Jamin W. Collins To be nobody but yourself when the whole world is trying it's best night and day to make you everybody else is to fight the hardest battle any human being will fight. -- E.E. Cummings export.gz Description: Binary data
Re: Managing Debian packages with Subversion
On Thu, Aug 28, 2003 at 02:13:51PM -0600, Jamin W. Collins wrote: Yes, that way you have a source structure devoid of the .svn subdirectories. I use the attached export script to ease the and build process. It assumes it is run from the project directory and that the orig source file and the svn working copy (in a directory called trunk) exist in the current directory. Guess it helps if I include the script that I referenced... -- Jamin W. Collins To be nobody but yourself when the whole world is trying it's best night and day to make you everybody else is to fight the hardest battle any human being will fight. -- E.E. Cummings export.gz Description: Binary data
Managing Debian packages with Subversion
Hi, I've read most of what is available about managing Debian packages with Subversion, but I've been unable to find a comprehensive answer to the following question: How do I manage a new Debian package with Subversion? The things I've found only scratch the surface a bit, assume a level of knowledge beyond mine, or assume you're migrating from CVS and already have made several initial decisions. Also, I'm hoping to include everything in the SVN repository, not just the debian/ directory. I'm not considering Debian native packages, but regular packages that have an upstream tarball. Here's what I have so far: * The repository will have a structure where the root contains the trunk/, branches/ and tags/ directories. * Projects will start their life in trunk/project, branches being stored in branches/project-branchname and tags in tags/. * Initial package creation will follow the following pattern: * Download upstream source and unpack, * debianize by dh_make, * integrate into SVN somehow. * Development cycle will follow the following pattern: * Check out HEAD from SVN, * make the required changes, * commit. Nothing special here. * When making a release... svn export? * When upgrading to a newer upstream version... how? I stress again that I'm looking for a solution that stores *everything* in SVN, not only the debian/ directory. Perhaps someone with greater insight could fill in the list above, and hopefully this thread will end with a nice description of how to accomplish the task. Cheers, -- Fabian Fagerholm [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
