Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
I don't think I did fully understand the interaction between your build system and your devs, as normally build systems like Jenkins, Hudson etc. run on a separate server fully automated and per default just don't commit anything, therefore it's very unimportant if the working copy they are working on is flagged as changed in any way or not. So your build system is used on the devs machines and works on the same working copy as your devs work on that you get a problem with svn status? Maybe I called our system the wrong name. It is a homemade system, and here are some of the things that it does: 1) Each project is specified by a custom configuration file, which lists the source files to be compiled using different tools and sets up project-specific options. These configuration files are versioned and may contain conditional options based on target platform and other global options. 2) A developer chooses from a list which projects he wishes to work on, and edits a configuration file to indicate the platform he wishes to compile to, build flags and several other options. These local options are specified by locally modifying a versioned template file. The modified file is marked as ignored, no problems here. 3) The system checks out or updates all relevant code from the repository, including the selected projects and, depending on the local options, other projects on which they depend, or that depend on them. This is done with svn sparse directories, no problems either. 4) The system automatically generates platform- and option-dependent files, such as makefiles on Linux and Visual Studio project and solution files on Windows. These are marked as ignored. 5) Before compilation, several tools produce some automatically-generated source code, such as bindings between two programming languages. These files can be different depending on the target platform and other options, and there can be quite a lot of them. They should also be marked as ignored, but there is no naming pattern or specific location for them, which makes it hard to use global ignore patterns. Since the automatically-generated files are often #included in other non-automatic source files, the developer is free to choose the location and naming which are best for him. The names and locations are specified in the project configuration files. 6) After compilation, the system copies some files around, such as exported library headers and DLL dependencies of the generated executables. 7) The system also automatically generates installation scripts and executables for the end-users. Number 5 above is probably the most problematic case. We _could_ try to impose a specific location and naming pattern for those files, but this could affect headers used by client developers as well, so we would rather avoid that if possible. Which subversion clients do you use? TortoiseSVN in Windows, just the command line in Unix.
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
Guten Tag Vitor Barata, am Dienstag, 30. Oktober 2012 um 06:58 schrieben Sie: Maybe I called our system the wrong name. It is a homemade system, and here are some of the things that it does: Just to be clear: What you call your build system is used directly by the devs and something we can think of as some kind of assistant to get pre configured working copies, right? The devs start this assistant, choose whatever fits their need and afterwards it is checking out, setting up and compiling things etc. and during that process svn working copies are created with versioned files with local modifications which should be ignored in svn status and comparable operations. If I understood correctly and you don't have any naming or directory layout convention to ignore the best chance in my opinion is your build system, as if it knows what it does automatically and should be ignored it should get capable of providing that information to svn. It could update the user wide configuration for ignore patterns of Subversion or specify paths to exclude for Tortoise etc. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
Just to be clear: What you call your build system is used directly by the devs and something we can think of as some kind of assistant to get pre configured working copies, right? The devs start this assistant, choose whatever fits their need and afterwards it is checking out, setting up and compiling things etc. and during that process svn working copies are created with versioned files with local modifications which should be ignored in svn status and comparable operations. That is almost correct. You got the nature of the system right, but it does not create versioned files with local modifications. It creates several unversioned files, which should be ignored as well. These files do not follow any naming or directory layout convention, their names and locations are freely specified as part of each project's specification, which is kept in versioned per-project configuration files. The unversioned files are the main problem. If I understood correctly and you don't have any naming or directory layout convention to ignore the best chance in my opinion is your build system, as if it knows what it does automatically and should be ignored it should get capable of providing that information to svn. It could update the user wide configuration for ignore patterns of Subversion or specify paths to exclude for Tortoise etc. Yes, indeed, the system knows exactly what should be ignored given the current in-development projects and other configuration options. With CVS, it marked these files as ignored by locally editing the .cvsignore file inside each folder, which did not have any undesired effect. Now, with SVN, our first idea was to locally edit each folder's svn:ignore property. However, since svn:ignore is itself versioned, every folder is always inconveniently marked as locally modified (property change only) by SVN, because the list of ignored files is different depending on the chosen projects and configuration options. And now we're back at my first post, self-quoted below. I can't help but think that there should be an unversioned list of ignored file patterns (.svnignore local file or svn:localignore property or something like that) as well as the versioned svn:ignore property. (self-quote) The only solution I could think of (to be able to keep a somewhat stable svn:ignore property) would be to populate the svn:ignore property with every file that could ever be automatically generated by the build system when operating in each and every local configuration scenario. Even then, however, there is still some unnecessary hassle: - For the property to be automatically updated, the build system would have to simulate every possible configuration (lots of possibilities) to update the svn:ignore property, which is a lot more work that working only with the single, currently selected configuration. - The system could get the current svn:ignore property and only add new items to it based on the currently selected configuration, but this may result in unnecessarily bloated svn:ignore properties over time, since no-longer-generated files would never be removed from the list. - If the property is to be managed manually, then the developers must remember to always update the svn:ignore property themselves every time a new file is automatically generated or an old file ceases to be generated. This is error-prone and requires every developer to know about the internals of the build system. The second option above is the strongest solution in my opinion, but it is still more complex than a simple .svnignore file (or an extra unversioned ignore property), and yet results in a not-as-good behavior.
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
Guten Tag Vitor Barata, am Dienstag, 30. Oktober 2012 um 14:54 schrieben Sie: I can't help but think that there should be an unversioned list of ignored file patterns (.svnignore local file or svn:localignore property or something like that) as well as the versioned svn:ignore property. But there's none and unless you code this feature on your own you won't get it anytime soon, I guess. The only thing left is your build system and the already present configuration for the used Subversion client and TortoiseSVN. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
On Tue, Oct 30, 2012 at 03:35:46PM +0100, Thorsten Schöning wrote: Guten Tag Vitor Barata, am Dienstag, 30. Oktober 2012 um 14:54 schrieben Sie: I can't help but think that there should be an unversioned list of ignored file patterns (.svnignore local file or svn:localignore property or something like that) as well as the versioned svn:ignore property. But there's none and unless you code this feature on your own you won't get it anytime soon, I guess. The only thing left is your build system and the already present configuration for the used Subversion client and TortoiseSVN. Mit freundlichen Grüßen, Thorsten Schöning Is editing the global-ignores setting in the Subversion configuration file, as already suggested by Thorsten in an earlier post, not an option? That's currently the only unversioned place where ignore settings can be configured. See http://svnbook.red-bean.com/en/1.7/svn.advanced.confarea.html#svn.advanced.confarea.opts.config Can the build system output its files into a subdirectory with a known name which can then be ignored? It seems your use case is rather exceptional. I've rarely seen the need to ignore arbitrary files with names that aren't known in advance. However, it's clear that the current ignore system isn't well suited for this case, so maybe we should extend Subversion in some way to account for this.
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
On Tue, Oct 30, 2012 at 01:57:09PM -0200, Vitor Barata wrote: A large part of the automatically-generated files contain source-code for language binding (Lua to C, Java to C and others). The system generates these files with .c/.cpp/.h extensions (so they can't be ignored based on the extension), but it does not enforce any convention on their names or on which directory they should go, this is freely configurable in custom project configuration files. This means that right now there is no global pattern that could be used in the global-ignores subversion setting. We do exactly the same thing in Subversion itself, and we do have ignore patterns like *.c and so on within the bindings source directories. There doesn't seem to be any problem with this in our project. This command lists the patterns we use: svn propget -v -R svn:ignore https://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings I'm not quite sure if you really understand the impact of svn:ignore. What behaviour are you expecting from the svn:ignore mechanism exactly, and how would the expected behaviour impact your workflow? Just to get a common misconception out of the way: Note that you can add files that match an ignore pattern to version control if you really want to. All ignore patterns do is suppress files from appearing in the output of 'svn status'. And some IDEs use ignore patterns to prevent some unversioned files from being suggested for commit but they should still allow such files to be added to version control. It seems your use case is rather exceptional. I've rarely seen the need to ignore arbitrary files with names that aren't known in advance. However, it's clear that the current ignore system isn't well suited for this case, so maybe we should extend Subversion in some way to account for this. The simplest and cleanest way I see to extend Subversion to account for this is to provide another special property, say svn:localignore, that would work exactly like svn:ignore, but that would be unversioned, so that it could be altered by automation systems like ours without making directories be marked as modified. Based on past experience, I would guess that this is probably not as simple as you imagine it to be, unfortunately. You are talking about a major feature addition, not some small fix. Subversion has no concept of an unversioned property yet. Such a concept would need to be added first which would require some design and implementation effort. A lot of work would need to be done to make sure this new kind of property interacts well with all other features of the system (status, diff, commit, update, ...). That means writing a lot regression tests to start with. This might turn out to be about as simple as the file external feature turned out to be. A very simple idea on the surface but very hard to get working correctly in all use cases, and we ended up frantically fixing a lot of new bugs related to the feature over the course of several months.
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
I'm not quite sure if you really understand the impact of svn:ignore. What behaviour are you expecting from the svn:ignore mechanism exactly, and how would the expected behaviour impact your workflow? We want the unversioned files to not show in svn status or in tortoiseSVN's Add command when invoked on directories which contain unversioned files. I'll talk to other devs to see if there are other expected behaviors. Based on past experience, I would guess that this is probably not as simple as you imagine it to be, unfortunately. You are talking about a major feature addition, not some small fix. Subversion has no concept of an unversioned property yet. Such a concept would need to be added first which would require some design and implementation effort. A lot of work would need to be done to make sure this new kind of property interacts well with all other features of the system (status, diff, commit, update, ...). That means writing a lot regression tests to start with. This might turn out to be about as simple as the file external feature turned out to be. A very simple idea on the surface but very hard to get working correctly in all use cases, and we ended up frantically fixing a lot of new bugs related to the feature over the course of several months. Alright, I got the picture. Don't worry, I'll look deeper into the matter and make a good effort to make things work nicely on our side before asking for such a feature. See below. We do exactly the same thing in Subversion itself, and we do have ignore patterns like *.c and so on within the bindings source directories. There doesn't seem to be any problem with this in our project. This command lists the patterns we use: svn propget -v -R svn:ignore https://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings The problem is that the C files automatically generated for bindings are often mixed with standard versioned C files... However, after further inspection here, it seems to me that the binding files could indeed be put in a separate directory without requiring any code to be changed. My main concern is with the exported header files, but their names don't seem to vary much with the target platform and other compilation options in our system. Maybe I can tell Subversion to ignore the headers individually and also a special bindings directory containing the implementations. I'm going to investigate this further, maybe there is a way to adapt our system without requiring code changes after all. I'll report here soon.
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
On Tue, Oct 30, 2012 at 12:55 PM, Vitor Barata vbar...@gmail.com wrote: The problem is that the C files automatically generated for bindings are often mixed with standard versioned C files... However, after further inspection here, it seems to me that the binding files could indeed be put in a separate directory without requiring any code to be changed. If they are automatically generated, why not automatically delete them when finished with them? Or, you might look into using a distributed build automation like Jenkins so the builds would happen in a separate workspace and the cruft doesn't get in the way of developer commits. -- Les Mikesell lesmikes...@gmail.com
RE: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
I have searched for this issue in several places, but none of the feature requesters presented a scenario like mine, and the overall conclusion was that the svn:ignore property was sufficient. However, I stand by the opinion that a local .svnignore file is also necessary (or some kind of _unversioned_ svn:localignore property). So, I'd like to see some additional opinions on the matter. Details follow. I have a build system that allows me to compile C/C++ code from a single repository for a number of different platforms, including several flavors of 32- and 64-bit Windows, Linux and MAC. During setup and compilation, the system generates several local files, which are different depending on the selected target platform, the selected in-development projects, and some other local configurations. These files are unversioned and should be ignored by the versioning tool. Until some time ago, I used CVS as my versioning tool, and the .cvsignore file worked perfectly. The build system knows what files will be generated for the current configuration, so it just created the .cvsignore file and filled it with the names of those files. Now, I have migrated my repository to SVN, and I am having trouble with the fact that the svn:ignore property is versioned. Since the list of ignored files varies with the local configuration, it is usually different for each developer in my team, and the source-code folders are always marked by SVN as modified, which is kind of inconvenient. The only solution I could think of (to be able to keep a somewhat stable svn:ignore property) would be to populate the svn:ignore property with every file that could ever be automatically generated by the build system when operating in each and every local configuration scenario. Even then, however, there is still some unnecessary hassle: - For the property to be automatically updated, the build system would have to simulate every possible configuration (lots of possibilities) to update the svn:ignore property, which is a lot more work that working only with the single, currently selected configuration. - The system could get the current svn:ignore property and only add new items to it based on the currently selected configuration, but this may result in unnecessarily bloated svn:ignore properties over time, since no-longer- generated files would never be removed from the list. - If the property is to be managed manually, then the developers must remember to always update the svn:ignore property themselves every time a new file is automatically generated or an old file ceases to be generated. This is error-prone and requires every developer to know about the internals of the build system. The second option above is the strongest solution in my opinion, but it is still more complex than a simple .svnignore file (or an extra unversioned ignore property), and yet results in a not-as-good behavior. So, I would like to know if anyone has a better idea, and/or if anyone agrees that a local .svnignore file (or an extra unversioned svn:ignore property) has a good use after all. Thanks for the attention, Vitor A couple of thoughts. Each developer for their specific platform could have a global ignores files that ignores the files that are specific to their platform/tool. You can generate a global ignore file since it is just a text file and place it in the correct location. Configure your build system and tools to place all build artifacts into an ignored work folder and copy files that are versioned into their correct locations. If the build is doing the add then it could create the svn:ignore property, do the add and then remove the property. Just some other possiblitites. BOb
Re: SVN offers svn:ignore property, but I also need some kind of .svnignore local file (or an extra unversioned version of svn:ignore)
Guten Tag Vitor Barata, am Montag, 29. Oktober 2012 um 16:56 schrieben Sie: So, I would like to know if anyone has a better idea, and/or if anyone agrees that a local .svnignore file (or an extra unversioned svn:ignore property) has a good use after all. I don't think I did fully understand the interaction between your build system and your devs, as normally build systems like Jenkins, Hudson etc. run on a separate server fully automated and per default just don't commit anything, therefore it's very unimportant if the working copy they are working on is flagged as changed in any way or not. So your build system is used on the devs machines and works on the same working copy as your devs work on that you get a problem with svn status? Which subversion clients do you use? If I read Windows I think of TortoiseSVN, which can be configured to exclude special folders as being marked as modified. Besides that, there's a global-ignores property for clients, too: [miscellany] ### Set global-ignores to a set of whitespace-delimited globs ### which Subversion will ignore in its 'status' output, and ### while importing or adding files and directories. ### '*' matches leading dots, e.g. '*.rej' matches '.foo.rej'. # global-ignores = *.o *.lo *.la *.al .libs *.so *.so.[0-9]* *.a *.pyc *.pyo # *.rej *~ #*# .#* .*.swp .DS_Store Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
