Re: Svn externals question
On Mon, Nov 08, 2010 at 10:24:03AM -, Hutchinson, Steve (UK) wrote: Do it the other way: Store your component configuration in a versioned file or even a database, and write a script to configure svn:externals properties based on that data. Maybe even add an automated check into the mix that makes sure the svn:externals in the repository's HEAD revision are in sync with your primary externals configuration source. I think unfortunately this is where you lose me :) I am not too sure why this is too different from the other way. In both situations I end up with a set of project folders with svn:externals on them. but the latter I say have one top level externals.lst and associated script file that is used to apply them. Probably a reflection of my knowledge level but not clear where the gain has come from ? It depends on what you need to do with information about your modules. It also depends on scale. If svn propget -R output is sufficient and fast enough for you, then you don't need a separate database of externals information. In this case you simply haven't scaled up your usage of externals to the point where propget -R becomes inhibiting. E.g. if you need tool support to configure and compose your modules or variants because you have a lot of them (say, various kinds of embedded devices built from common parts), having the configuration data in svn properties means that an application has to crawl the repository to figure out which components are used where. Also consider that you need might to crawl more than one revision to find an answer you're looking for. A database can instantly tell you things like this component is no longer used, and was last used in devices A, B and C. if fed a proper query. If such information is hidden in externals in past revisions, you're putting unnecessary load on your Subversion server by trying to track down the right properties. Stefan
RE: Svn externals question
Stefan, Your point well understood now. Thanks for the clarification, appreciated. Steve H -Original Message- From: Stefan Sperling [mailto:s...@elego.de] Sent: 08 November 2010 17:01 To: Hutchinson, Steve (UK) Cc: users@subversion.apache.org Subject: Re: Svn externals question SPECIAL MBDA ALERT - SECURITY INCREASED FOR WEEK COMMENCING 08/11/10 PLEASE TAKE EXTRA CARE NOT TO CLICK ON HYPERLINKS WHERE YOU DO NOT KNOW THE SENDER ANY SUSPICIOUS EMAIL, PLEASE FORWARD TO ftn.VIRUS CONTROL *** WARNING *** This mail has originated outside your organization, either from an external partner or the Global Internet. Keep this in mind if you answer this message. On Mon, Nov 08, 2010 at 10:24:03AM -, Hutchinson, Steve (UK) wrote: Do it the other way: Store your component configuration in a versioned file or even a database, and write a script to configure svn:externals properties based on that data. Maybe even add an automated check into the mix that makes sure the svn:externals in the repository's HEAD revision are in sync with your primary externals configuration source. I think unfortunately this is where you lose me :) I am not too sure why this is too different from the other way. In both situations I end up with a set of project folders with svn:externals on them. but the latter I say have one top level externals.lst and associated script file that is used to apply them. Probably a reflection of my knowledge level but not clear where the gain has come from ? It depends on what you need to do with information about your modules. It also depends on scale. If svn propget -R output is sufficient and fast enough for you, then you don't need a separate database of externals information. In this case you simply haven't scaled up your usage of externals to the point where propget -R becomes inhibiting. E.g. if you need tool support to configure and compose your modules or variants because you have a lot of them (say, various kinds of embedded devices built from common parts), having the configuration data in svn properties means that an application has to crawl the repository to figure out which components are used where. Also consider that you need might to crawl more than one revision to find an answer you're looking for. A database can instantly tell you things like this component is no longer used, and was last used in devices A, B and C. if fed a proper query. If such information is hidden in externals in past revisions, you're putting unnecessary load on your Subversion server by trying to track down the right properties. Stefan This email and any attachments are confidential to the intended recipient and may also be privileged. If you are not the intended recipient please delete it from your system and notify the sender. You should not copy it or use it for any purpose nor disclose or distribute its contents to any other person. MBDA UK Limited, a company registered in England and Wales, registration number 3144919 whose registered office is at Six Hills Way, Stevenage, Hertfordshire, SG1 2DA, England. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Svn externals question
Hi, Currently we are attempting to use svn externals to help build various projects from what I would call a few reuse repositories. We are attempting to be structured as to what level of design hierarchy we apply the properties but sometimes when we inherit a design people can spend a bit of time trying to identify where externals have been used. Is there a simple way of identifying in a structure folders that have external properties, come to think of it maybe any form of property ? Thanks for any help. Regards Steve Hutchinson FPGA Group Leader This email and any attachments are confidential to the intended recipient and may also be privileged. If you are not the intended recipient please delete it from your system and notify the sender. You should not copy it or use it for any purpose nor disclose or distribute its contents to any other person. MBDA UK Limited, a company registered in England and Wales, registration number 3144919 whose registered office is at Six Hills Way, Stevenage, Hertfordshire, SG1 2DA, England. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Svn externals question
In a checked-out working copy, svn status marks directories loaded by virtue of svn:externals with an 'X'. Other props, and finding even that one from the repository, requires scripting a loop to use svn plist or similar, I believe. Sent from my iPhone On Nov 5, 2010, at 9:50 AM, Hutchinson, Steve (UK) steven.hutchin...@mbda-systems.com wrote: Hi, Currently we are attempting to use svn externals to help build various projects from what I would call a few reuse repositories. We are attempting to be structured as to what level of design hierarchy we apply the properties but sometimes when we inherit a design people can spend a bit of time trying to identify where externals have been used. Is there a simple way of identifying in a structure folders that have external properties, come to think of it maybe any form of property ? Thanks for any help. Regards Steve Hutchinson FPGA Group Leader This email and any attachments are confidential to the intended recipient and may also be privileged. If you are not the intended recipient please delete it from your system and notify the sender. You should not copy it or use it for any purpose nor disclose or distribute its contents to any other person. MBDA UK Limited, a company registered in England and Wales, registration number 3144919 whose registered office is at Six Hills Way, Stevenage, Hertfordshire, SG1 2DA, England. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Svn externals question
On Fri, Nov 05, 2010 at 01:49:03PM -, Hutchinson, Steve (UK) wrote: Hi, Currently we are attempting to use svn externals to help build various projects from what I would call a few reuse repositories. We are attempting to be structured as to what level of design hierarchy we apply the properties but sometimes when we inherit a design people can spend a bit of time trying to identify where externals have been used. Is there a simple way of identifying in a structure folders that have external properties, come to think of it maybe any form of property ? The designers of the externals feature envisioned maybe a handful of external library dependencies that don't vary much over time. These are automatically pulled into a working copy, much like an automated svn checkout. But the design doesn't account for what happens when people start using svn:externals for variant management or large-scale component reuse. If you're pulling together project components from externals in various combinations, like lego blocks, or simply have many externals, don't use the svn:externals properties as the primary source of your configuration data. Do it the other way: Store your component configuration in a versioned file or even a database, and write a script to configure svn:externals properties based on that data. Maybe even add an automated check into the mix that makes sure the svn:externals in the repository's HEAD revision are in sync with your primary externals configuration source. You can query a file or a database easily to find out which components are used where. But svn properties haven't been designed for this use case. You cannot query a Subversion repository like you can query a database. Well, you could crawl the repository, but that's quite slow. Hope this helps, Stefan
Re: Svn externals question
If you do switch to the approach Stefan suggests (and I agree, it's probably more satisfactory) you also might want to use svn switch instead of the svn:externals. You'll still have the auditable, versioned definition of your canonical configuration (in the form of the script), but also there will be more freedom for variations, such as while preparing a new configuration. Sent from my iPhone On Nov 5, 2010, at 10:09 AM, Stefan Sperling s...@elego.de wrote: On Fri, Nov 05, 2010 at 01:49:03PM -, Hutchinson, Steve (UK) wrote: Hi, Currently we are attempting to use svn externals to help build various projects from what I would call a few reuse repositories. We are attempting to be structured as to what level of design hierarchy we apply the properties but sometimes when we inherit a design people can spend a bit of time trying to identify where externals have been used. Is there a simple way of identifying in a structure folders that have external properties, come to think of it maybe any form of property ? The designers of the externals feature envisioned maybe a handful of external library dependencies that don't vary much over time. These are automatically pulled into a working copy, much like an automated svn checkout. But the design doesn't account for what happens when people start using svn:externals for variant management or large-scale component reuse. If you're pulling together project components from externals in various combinations, like lego blocks, or simply have many externals, don't use the svn:externals properties as the primary source of your configuration data. Do it the other way: Store your component configuration in a versioned file or even a database, and write a script to configure svn:externals properties based on that data. Maybe even add an automated check into the mix that makes sure the svn:externals in the repository's HEAD revision are in sync with your primary externals configuration source. You can query a file or a database easily to find out which components are used where. But svn properties haven't been designed for this use case. You cannot query a Subversion repository like you can query a database. Well, you could crawl the repository, but that's quite slow. Hope this helps, Stefan
Re: Svn externals question
On Fri, Nov 05, 2010 at 11:03:58AM -0400, Jack Repenning wrote: If you do switch to the approach Stefan suggests (and I agree, it's probably more satisfactory) you also might want to use svn switch instead of the svn:externals. You'll still have the auditable, versioned definition of your canonical configuration (in the form of the script), but also there will be more freedom for variations, such as while preparing a new configuration. Right. But note that switched subtrees cannot come from a different repository.
Re: Svn externals question
On Fri, Nov 5, 2010 at 3:09 PM, Stefan Sperling s...@elego.de wrote: On Fri, Nov 05, 2010 at 01:49:03PM -, Hutchinson, Steve (UK) wrote: Hi, Currently we are attempting to use svn externals to help build various projects from what I would call a few reuse repositories. We are attempting to be structured as to what level of design hierarchy we apply the properties but sometimes when we inherit a design people can spend a bit of time trying to identify where externals have been used. Is there a simple way of identifying in a structure folders that have external properties, come to think of it maybe any form of property ? The designers of the externals feature envisioned maybe a handful of external library dependencies that don't vary much over time. These are automatically pulled into a working copy, much like an automated svn checkout. But the design doesn't account for what happens when people start using svn:externals for variant management or large-scale component reuse. If you're pulling together project components from externals in various combinations, like lego blocks, or simply have many externals, don't use the svn:externals properties as the primary source of your configuration data. Do it the other way: Store your component configuration in a versioned file or even a database, and write a script to configure svn:externals properties based on that data. Maybe even add an automated check into the mix that makes sure the svn:externals in the repository's HEAD revision are in sync with your primary externals configuration source. You can query a file or a database easily to find out which components are used where. But svn properties haven't been designed for this use case. You cannot query a Subversion repository like you can query a database. Well, you could crawl the repository, but that's quite slow. Hope this helps, Stefan For finding the folders that have the svn:externals property, how about: svn propget -R svn:externals TARGET where TARGET can be a working copy path or a url. For a url it will not be fast, but it works (for a working copy it works quite fast). I do agree with Stefan though, that it's best to have another canonical source for this kind of configuration data, especially if it starts becoming more and more complex, and to derive the externals configuration from that. Cheers, -- Johan
Re: Svn externals question
On Fri, Nov 5, 2010 at 9:49 AM, Hutchinson, Steve (UK) steven.hutchin...@mbda-systems.com wrote: Is there a simple way of identifying in a structure folders that have external properties, come to think of it maybe any form of property ? Not 100% clear what you're looking for. You could be looking for one of two things: 1). You have a project, and it has svn:externals set on certain folders. You want to find those folders. That's fairly easy to do with the svn propget. Go to the root of your project and run: $ svn propget -v -R svn:externals That will show you all the directories where svn:externals are set and directories are being pulled in. 2). You have a bunch of external projects, and want to know in what other projects they're being used. This is much more difficult since there's no way to see what projects are using a particular directory as an external project. The only thing I can think of is to run the svn propget on the entire repository and parse the output. -- David Weintraub qazw...@gmail.com
Re: Svn externals question
On Fri, Nov 5, 2010 at 10:09 AM, Stefan Sperling s...@elego.de wrote: The designers of the externals feature envisioned maybe a handful of external library dependencies that don't vary much over time. These are automatically pulled into a working copy, much like an automated svn checkout. But the design doesn't account for what happens when people start using svn:externals for variant management or large-scale component reuse. If you're pulling together project components from externals in various combinations, like lego blocks, or simply have many externals, don't use the svn:externals properties as the primary source of your configuration data. The big problem with svn:externals is that they don't version very well. Imagine I have a project called foo, and in the root directory, I set an svn:external property to pull in the trunk of module bar that contains certain routines I'm using: $ svn propset svn:externals bar /externals/bar/trunk When foo is ready for Release 1.0, I create a tag: $ svn copy svn://subversion/foo/trunk svn://subversion/foo/tags/REL-1.0 Everything is fine and dandy, but we found a few bugs, so we're going to create a Release 1.1 to fix those bugs. I create my 1.1 branch: $ svn copy svn://subversion/foo/tags/REL-1.0 svn://subversion/foo/branches/1.1 But, there's a problem. I discover that what is tagged as REL-1.0 isn't the code I released. In fact, all the difference between what has been tagged and what was released are in that bar subdirectory that was created by my svn:externals property. The problem is that the external directories themselves aren't tagged or branched when I did my tag or branch. Instead, the svn:external property itself was versioned, and if you have that pointing to just the branch or the trunk without a version number, that external directory can keep changing in the tag. You can partially solve that problem by making sure your svn:external property points to a specific tag and/or a specific version of the external directory. But, then you have to start keeping the svn:external tags up to date which can be a headache if you have a lot of them. When I first started using Subversion, I though svn:externals was a great feature. But I quickly discovered what a mess they can create. I now take the approach of thinking of my externals as releasable components that are also versioned and released, and I put them into a release repository where various projects can import a specific revision of that component. This makes it much easier for developers to track what particular version of that component they're using. -- David Weintraub qazw...@gmail.com
Re: Svn externals question
On Nov 5, 2010, at 11:54, David Weintraub wrote: The big problem with svn:externals is that they don't version very well. The problem is that the external directories themselves aren't tagged or branched when I did my tag or branch If this is important to you, then you should use the svncopy.pl script to create your tags because this is exactly what it takes care of for you.
