Re: Update-Only Checkout Enhancement
On 12.12.2013 08:30, Les Mikesell wrote: On Wed, Dec 11, 2013 at 11:01 PM, Ryan Schmidt subversion-20...@ryandesign.com wrote: On Dec 11, 2013, at 19:19, Les Mikesell wrote: Also, it would mean you would need a constant connection to the server to use a subversion working copy. That's hardly a problem these days, You apparently don’t try to work at the kinds of coffee shops I go to, where 50 college students are all watching youtube videos and making the network unbearable and Subversion’s ability to do some things offline very valuable to me. Some things... But not the things you really need to complete any amount of actual work - like updates and commits. You're forgetting diff. If you use Subversion daily, you've become so used to it being local that you can't appreciate how slow it would be without locally cached pristine copies. Anyway, this thread has gone way off any kind of reasonable topic. The short answer to the original poster is that Subversion is not intended to be a replacement for rsync. You can take that as a hint as to what the solution to your problem might be. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: how to svn merge with multiple revision numbers - quick question
On 12.12.2013 07:32, Zk W wrote: Hi All We have a few revision numbers on trunk that we like to block but they are not in consecutive order - i.e. rev numbers 1,5,8,10,11 We like to merge from trunk to a branch but would like to block them on the branch. What's the correct syntax for svn merge --record-only ? Is it svn merge -c 1,5,8,10,11 --record-only trunk/directory or svn merge -r 1:11 --record-only trunk/directory The correct syntax is svn help merge. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
RE: Update-Only Checkout Enhancement
From: Ben Reser [mailto:b...@reser.org] Absolutely, the answer here isn't a one size fits all. Nobody is objecting to the idea of allowing this. The problem is that the code is not designed to allow this and it's a ton of work to change that. I can think of a good 10 other things that would require the same level of effort that would improve things for a lot more people Should you get into doing this, I hope it will be highly configurable both from the server side and client side. I can definitely see running a checkout operation where you'd like to specify that for anything under a certain URL within your checkout, you don't want pristine (just what I presume would be sufficient metadata like dates+hashes to detect any changes that may occur). Or after fetching, you might want to prune out the pristines for a file or directory tree. You may or may not change the files afterward, and as folks have noted, you'd have to fetch down from the server in any commit operation, and you'd have to do a double-fetch of sorts (working base plus intended revision) in order to update in cases where such a file had been locally altered. Note that this pruning sort of behavior may require maintaining client-side metadata, perhaps similar to how changelists are client-side. You would also want to be able to configure on the server something like svn:no-pristines to say that by default there would be no pristine in any working copy that downloaded a given file/path, and then you'd need to do some opposite logic from any client that did want the pristines. This could be good when most people don't work in a given area, but you want to accommodate the few people who do by letting them demand the usual structure. And yes, it's icing on the cake, though I can certainly imagine using it. I have enough working copies that disk space is very much an issue for me. Once you no longer need a pristine there are a lot of potential scenarios that require different behaviors. So it's not even a simple matter of just changing the working copy library to support pristines not existing. It becomes thinking about how to deal with these scenarios (not that all of them need to be implemented immediately, but you probably want to not pigeon hole yourself into an implementation that doesn't support them). Some files don't delta very well and essentially change every bit when the file changes, so those files update and commit are better off just transmitting full text. Some files do delta fine but are just large, if you change the file you may want to run a diff and update/commit would benefit from deltas. If you have a file that deltas well should you download the full text from the server and produce a delta locally or just send the full text to the server? The answer to this depends on the network. If you're on a asymmetric connection where upload or download or different whichever method that sends the largest data over the fastest direction is better (e.g. a typical DSL or Cable internet connection where download is faster it may be far better to download full text and upload the delta you calculate). If you're on a symmetric network you just do whichever takes the least work (directly sending/retrieving full text). Due to laptops that came move between networks you also can't just configure the file on the server side (i.e. set a property). You do what you have to do based on what's in the WC (which is determined by server and client configurations). If you're committing without a pristine, I think you must either send the whole new file, or else re-receive the working base file from the server. That is, unless you do something like running checksums/hashes on fragments of the file to pinpoint changes, which you might do if the file is really large, or (my preference) if a particular svn: attribute was set on the file/folder. Of course for doing updates, if you can determine based on the file's metadata (checksum/hash info) that the file hasn't changed, then the usual diff should still function. What about compressing the file. If it compresses well then you could have your cake and eat it too... You save disk space and you still have the pristine. I don't think this gets you very far. Probably worth doing, but I wouldn't see it as the ultimate answer. The user may want us to retain the pristine data we retrieved to execute a command or they may just want us to destroy it immediately to limit disk space. Exactly, and that's where the client-side configuration I described above would come into play. For that matter the pristine store as of 1.7 doesn't even remove files. So if you're updating a working copy it will just continue to grow. As of 1.8 we use this to allow faster updates when you already have a copy of a version of a file in the pristine store. So we need some sort of garbage collector for the pristine store even
Re: Update-Only Checkout Enhancement
On Thu, Dec 12, 2013 at 4:02 AM, Branko Čibej br...@wandisco.com wrote: Some things... But not the things you really need to complete any amount of actual work - like updates and commits. You're forgetting diff. If you use Subversion daily, you've become so used to it being local that you can't appreciate how slow it would be without locally cached pristine copies. But (a) it is trivial to make your own snapshot copy of file versions if you expect to need them (as you must anyway for any offline state except 'as checked out' and 'completely current'. And (b), I'm as likely to want to diff against a different branch or someone else's subsequent commit as my own pristine version (and no, that isn't unreasonably slow...).Also, most of this discussion is related to scenarios where active development is not happening - like the jenkins build server example. Anyway, this thread has gone way off any kind of reasonable topic. The short answer to the original poster is that Subversion is not intended to be a replacement for rsync. You can take that as a hint as to what the solution to your problem might be. But it _was_ intended to be a replacement for cvs, which has nowhere near the client-side resource requirements. -- Les Mikesell lesmikes...@gmail.com
Re: Update-Only Checkout Enhancement
On 12/10/2013 8:45 PM, Mark Kneisler wrote: I have several environments where I’d like to use a SVN checkout, but where I’d never ever want to make changes to the files or perform a commit. For these environments, I’d only want to perform an update or an update to revision. In cases where you do not want a .svn directory and you are using Linux, take a look at FSVS: http://fsvs.tigris.org/ This is a command line tool that works very similar to the svn command-line tool and talks to an SVN repository. We make heavy use of it to version-control our Linux servers (especially the files under /usr/local, /boot and /etc). The big difference over using FSVS vs SVN on the Linux box is that FSVS does not create a .svn folder in the root. I don't know off-hand if FSVS can be used in Cygwin under Windows.
Re: Update-Only Checkout Enhancement
On 12/11/2013 2:19 PM, Bob Archer wrote: On 11.12.2013 17:21, Mark Kneisler wrote: I think making the pristine files optional would work for me. Here’s an idea. Instead of having pristine copies of all files, how about adding to the pristine directory only when a file is changed? You know, that's a great idea! I wonder why we never thought of it ourselves? :) Wouldn't that mean that you need to have some daemon service (or file watcher or something) running to determine if a file is modified? Also, it would mean you would need a constant connection to the server to use a subversion working copy. Not necessarily. Take a look at how FSVS does its magic. http://fsvs.tigris.org/ It functions in a similar manner to the svn command-line tool, but works without requring a .svn folder. Which is why I prefer it for doing version control of system configuration files on a Linux server.
Re: Tools for projecting growth of repository storage?
On 12/2/2013 7:58 PM, Eric Johnson wrote: Anyone have a suggestion for a tool that projects the growth of repository storage. I've got repos taking over 75% of a disk volume, and I'm curious to project out when I'll need new storage. Obviously, this is approximate, but has anyone got a tool for it? Eric. We keep our repositories on a dedicated file system (ext4) and run collectd on the box to track file system space usage (the df plugin). Combine that with a graphing tool for collectd that can read the RRD files (such as the web-based CGP front-end) and we get nice pretty charts. http://imgur.com/xDZ9BGu As you can see in Week 27-29, we had some runaway growth which alerted me that I needed to take a look at what was being automatically committed. In our case, it was FSVS doing automated commits of a Linux box where we should have ignored/excluded some additional directories. When looking at my quarterly graph (13 weeks), CGP gives me numbers like: Used (Minimum) 96.9GB Used (Last) 99.2GB - which means I have only seen 2.3GB of growth over 13 weeks, or about 10GB per year at current rate of growth. We also run a small script each day that checks the file systems and sends an alert if any file system is over 75% full.
help with svn:ignore
Hi All, I am using Subversion 1.6.11-10 on Red Hat Enterprise 5.8. The repository is strictly for RANCID and I recently renamed several files in my repository with the svn move command. After making these changes I found that I can no longer see the file history for the renamed files in my repository viewer, OpenGrok. I have checked out an updated copy of the repository and cleared the OpenGrok caches but still cannot see historical revisions. I executed an svn log for the repo with the following results. r2044 | rancid | 2013-12-11 17:38:10 + (Wed, 11 Dec 2013) | 1 line new router r2043 | rancid | 2013-12-10 22:31:13 + (Tue, 10 Dec 2013) | 1 line updates r2042 | rancid | 2013-12-10 20:56:41 + (Tue, 10 Dec 2013) | 1 line updates r2041 | rancid | 2013-12-10 20:34:13 + (Tue, 10 Dec 2013) | 1 line set svn:ignores r3 | rancid | 2013-12-10 19:04:29 + (Tue, 10 Dec 2013) | 1 line new Newer revisions show updates or new router (new file) and there is history for all revisions past r2041. I executed svn propdel svn:ignore --revprop -r 2041 file:///opt/rancid/archives/SVN/CISCO/file:///\\opt\rancid\archives\SVN\CISCO\ and received a message of property 'svn:ignore' deleted from repository revision 2041. I then update my RANCID working copy on the server with svn update and check out the changes for OpenGrok but I still cannot see historical changes. The changes are in the repository because I can execute svn cat file:///opt/rancid/archives/SVN/CISCO/configs/router1 -r1663 to see the historical file. Can someone help me find a way to remove the svn:ignore that has been set at revision 2041? Thank you, Scotty
Re: Update-Only Checkout Enhancement
Bob Archer wrote on Wed, Dec 11, 2013 at 21:05:46 +: There may be some transactional file systems that allow you to get an event before the modification is committed to the file system so you can access the original copy, but I think they are few and far between. And also would require new code on svn's side to be written. There are, though, deduplicating filesystems, that store a file just once if there are several copies of it. That means a file and its pristine won't consume twice the platter space, assuming the file doesn't use either svn:keywords or svn:eol-style=native. (I'm thinking of zfs. And BTW, the pristine store also deduplicates, so if two pristines have the same sha1, they're stored just once.)
RE: help with svn:ignore
I am using Subversion 1.6.11-10 on Red Hat Enterprise 5.8. The repository is strictly for RANCID and I recently renamed several files in my repository with the svn move command. After making these changes I found that I can no longer see the file history for the renamed files in my repository viewer, OpenGrok. I have checked out an updated copy of the repository and cleared the OpenGrok caches but still cannot see historical revisions. I executed an svn log for the repo with the following results. r2044 | rancid | 2013-12-11 17:38:10 + (Wed, 11 Dec 2013) | 1 line new router r2043 | rancid | 2013-12-10 22:31:13 + (Tue, 10 Dec 2013) | 1 line updates r2042 | rancid | 2013-12-10 20:56:41 + (Tue, 10 Dec 2013) | 1 line updates r2041 | rancid | 2013-12-10 20:34:13 + (Tue, 10 Dec 2013) | 1 line set svn:ignores r3 | rancid | 2013-12-10 19:04:29 + (Tue, 10 Dec 2013) | 1 line new Newer revisions show updates or new router (new file) and there is history for all revisions past r2041. I executed svn propdel svn:ignore --revprop -r 2041 file:///opt/rancid/archives/SVN/CISCO/ and received a message of property 'svn:ignore' deleted from repository revision 2041. I then update my RANCID working copy on the server with svn update and check out the changes for OpenGrok but I still cannot see historical changes. The changes are in the repository because I can execute svn cat file:///opt/rancid/archives/SVN/CISCO/configs/router1 -r1663 to see the historical file. Can someone help me find a way to remove the svn:ignore that has been set at revision 2041? You can't remove versioned properties. Changes to those properties are part of the history of the file/path just like the changes made to the file. It is not a revprop. That said, svn ingnore doesn't affect stuff that is in the repository already. So, I don't think that is your problem. Also, the property affects when stuff is added. When you did the move, did you commit both the ADD and the DELETE? You can verify by looking at the commit info. Does OpenGrock perhaps automatically include a --stop-on-copy for the log command? BOb
RE: help with svn:ignore
Hi Bob, I appreciate the reply. A few more details that may help. I wrote a Perl script to move the files from the old name to the new name. During the first round there were some issues with renaming so I had to restore a backup of the repository. I created a new directory then restored my dump file from the previous day. After restoring the dump I moved the restored directory into the correct path for RANCID then moved the logs and SVN directories. /usr/libexec/rancid/rancid-cvs svnadmin load /opt/rancid/archives/SVN/ /opt/svn_backups/restore.dump svn co file:///opt/rancid/archives/SVN/ mv archives archives.new mv SVN archives cd archives.new mv logs/ ../archives mv SVN/ ../archives I then moved the files with my Perl script. It appears from the output of svn log -v file:///opt/rancid/archives/SVN/CISCO/ that r1 - r3 is when RANCID created the repo using rancid-cvs and r2041 is where I moved the files. Maybe I am not restoring the repository correctly but RANCID needs a working copy locally on the server to run its jobs which is why I checked out the repo then moved the SVN and logs dir. I have another server that I also renamed files on using svn move but I moved the files one at a time instead of with a script. This server reports svn:ignores at revisions after moves were executed even though I did not restore a backup. Server 1 -- Restored backup Moved files with Perl script svn:ignores set Server 2 -- No backup restored Moved files one at a time svn:ignores set Are there some additional flags I should be using with svn move? When manually moving or moving with the script the command is the same svn move /path/old_name /path/new_name. I do not see any stop on copy in the svn log output that OpenGrok would pick up on. Thank you, Scotty -Original Message- From: Bob Archer [mailto:bob.arc...@amsi.com] Sent: Thursday, December 12, 2013 2:11 PM To: Hinote, Scotty (MSFC-IS40)[NICS]; users@subversion.apache.org Subject: RE: help with svn:ignore I am using Subversion 1.6.11-10 on Red Hat Enterprise 5.8. The repository is strictly for RANCID and I recently renamed several files in my repository with the svn move command. After making these changes I found that I can no longer see the file history for the renamed files in my repository viewer, OpenGrok. I have checked out an updated copy of the repository and cleared the OpenGrok caches but still cannot see historical revisions. I executed an svn log for the repo with the following results. r2044 | rancid | 2013-12-11 17:38:10 + (Wed, 11 Dec 2013) | 1 line new router r2043 | rancid | 2013-12-10 22:31:13 + (Tue, 10 Dec 2013) | 1 line updates r2042 | rancid | 2013-12-10 20:56:41 + (Tue, 10 Dec 2013) | 1 line updates r2041 | rancid | 2013-12-10 20:34:13 + (Tue, 10 Dec 2013) | 1 line set svn:ignores r3 | rancid | 2013-12-10 19:04:29 + (Tue, 10 Dec 2013) | 1 line new Newer revisions show updates or new router (new file) and there is history for all revisions past r2041. I executed svn propdel svn:ignore --revprop -r 2041 file:///opt/rancid/archives/SVN/CISCO/ and received a message of property 'svn:ignore' deleted from repository revision 2041. I then update my RANCID working copy on the server with svn update and check out the changes for OpenGrok but I still cannot see historical changes. The changes are in the repository because I can execute svn cat file:///opt/rancid/archives/SVN/CISCO/configs/router1 -r1663 to see the historical file. Can someone help me find a way to remove the svn:ignore that has been set at revision 2041? You can't remove versioned properties. Changes to those properties are part of the history of the file/path just like the changes made to the file. It is not a revprop. That said, svn ingnore doesn't affect stuff that is in the repository already. So, I don't think that is your problem. Also, the property affects when stuff is added. When you did the move, did you commit both the ADD and the DELETE? You can verify by looking at the commit info. Does OpenGrock perhaps automatically include a --stop-on-copy for the log command? BOb
Re: Tools for projecting growth of repository storage?
On Mon, Dec 2, 2013 at 7:58 PM, Eric Johnson e...@tibco.com wrote: Anyone have a suggestion for a tool that projects the growth of repository storage. I've got repos taking over 75% of a disk volume, and I'm curious to project out when I'll need new storage. Obviously, this is approximate, but has anyone got a tool for it? Subversion Edge has a feature that is sort of like this built-in. Basically it captures the amount of disk space used by your repositories and graphs them over time. It would be up to you to build projections based on what it shows you. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: help with svn:ignore
Guten Tag Hinote, Scotty (MSFC-IS40)[NICS], am Donnerstag, 12. Dezember 2013 um 21:38 schrieben Sie: I then moved the files with my Perl script. You should provide the lines which implement the rename and the commit of the deleted and newly added files. It appears from the output of svn log -v file:///opt/rancid/archives/SVN/CISCO/ that r1 - r3 is when RANCID created the repo using rancid-cvs and r2041 is where I moved the files. Did you use svn log on the files moved to get their history? Execute svn help log and have a look at --stop-on-copy. I have another server that I also renamed files on using svn move but I moved the files one at a time instead of with a script. This server reports svn:ignores at revisions after moves were executed even though I did not restore a backup. svn:ignore surely is not your problem, it doesn't affect the output or generation of history in any way. It's only purpose is to specify to ignore unadded files in a working copy and therefore prevent accidental commits of binary files etc. Are there some additional flags I should be using with svn move? Doesn't seem so, but it essential to commit both delete and add of the renamed file in the same commit. When manually moving or moving with the script the command is the same svn move /path/old_name /path/new_name. I do not see any stop on copy in the svn log output that OpenGrok would pick up on. stop-on-copy is an argument to svn log, you won't see it in any output, just the results may not be what you expect. Besides that, it seems that OpenGrok is able to handle renames, but your version may simply have bugs: https://github.com/OpenGrok/OpenGrok/issues/694 https://github.com/OpenGrok/OpenGrok/issues/4 Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Bug report on SVN 1.8.3 Cannot allocate Memory
Hi, We had subversion 1.8.3 installed on our test server. svn, version 1.8.3 (r1516576) compiled Aug 30 2013, 17:19:00 on x86_64-unknown-linux-gnu The OS on this server is OS Red Hat Enterprise Linux Server release 6.3 (Santiago) . Initially, testing 1.8.3 on this test server looks good but then we ran into this showstopper a few weeks ago. svn rm URL -m Remove test dir svn: E165002: Failed to start '/../../../hooks/start-commit' hook Looking into the subversion error_log and it appears it point to the memory [Mon Nov 18 17:28:53 2013] [error] [client 10.20.36.51] Can't start process '/data00start-commit': Cannot allocate memory [500, #12] Our Linux System Admin had applied different memory cache setting to the server but thus far we have no luck resolving this error. If you could please provide any insights on why and ho we could correct this issue, I'm truly appreciated your help/ Thanks in advance for your help Quyen Nguyen Quality Assurance Engineer Epsilon Targeting 2550 Crescent Drive, Lafayette, CO 80026 T 303-410-5542 F 303-420-5674 quyen.ngu...@epsilon.commailto:quyen.ngu...@epsilon.com www.epsilontargeting.comhttp://www.epsilontargeting.com/ [Description: C:\Documents and Settings\ctorok\Desktop\Aspen_Epsilon_Assets\Logos\E-Mail_Signature_Logos\Epsilon_logo_PMS_wTag_horiz.jpg] This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You. inline: image001.jpg
Re: Bug report on SVN 1.8.3 Cannot allocate Memory
On 12/12/13 2:12 PM, Nguyen, Quyen wrote: Looking into the subversion error_log and it appears it point to the memory [Mon Nov 18 17:28:53 2013] [error] [client 10.20.36.51] Can't start process '/data00start-commit': Cannot allocate memory [500, #12] That is an OS error. Subversion is providing you the Cannot allocate memory message as a convenience, it's not producing the error. Looking at the code I'd bet that fork() is returning ENOMEM on your system. Our Linux System Admin had applied different memory cache setting to the server but thus far we have no luck resolving this error. If you could please provide any insights on why and ho we could correct this issue, I’m truly appreciated your help/ Add more memory to the system or figure out why the kernel doesn't think it has enough memory. Might want to check your configured ulimits as well.
RE: Bug report on SVN 1.8.3 Cannot allocate Memory
Thank you for your input Ben. We will get with our System Admin and see what is going there. Quyen Nguyen -Original Message- From: Ben Reser [mailto:b...@reser.org] Sent: Thursday, December 12, 2013 3:37 PM To: Nguyen, Quyen; users@subversion.apache.org Cc: Vornbrock, Beth Subject: Re: Bug report on SVN 1.8.3 Cannot allocate Memory On 12/12/13 2:12 PM, Nguyen, Quyen wrote: Looking into the subversion error_log and it appears it point to the memory [Mon Nov 18 17:28:53 2013] [error] [client 10.20.36.51] Can't start process '/data00start-commit': Cannot allocate memory [500, #12] That is an OS error. Subversion is providing you the Cannot allocate memory message as a convenience, it's not producing the error. Looking at the code I'd bet that fork() is returning ENOMEM on your system. Our Linux System Admin had applied different memory cache setting to the server but thus far we have no luck resolving this error. If you could please provide any insights on why and ho we could correct this issue, I'm truly appreciated your help/ Add more memory to the system or figure out why the kernel doesn't think it has enough memory. Might want to check your configured ulimits as well. This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Bug report on SVN 1.8.3 Cannot allocate Memory
Guten Tag Nguyen, Quyen, am Donnerstag, 12. Dezember 2013 um 23:12 schrieben Sie: svn: E165002: Failed to start '/../../../hooks/start-commit' hook It may help to provide your hook's source to the list, just in case there's an error in it like a wrong shebang, wrong line endings or stuff like that which may prevent fork from working as expected. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Bug report on SVN 1.8.3 Cannot allocate Memory
We will get with our System Admin and see what is going there. The command dmesg might also give a clue on Linux systems. regards Henrik
