Space optimisation of a local checkout
Hello, I keep a local svn checkout in sync with a svn repository on the server (without making any local changes). I probably created the repository with subversion 1.6 (and also did the checkout with the same version) and later upgraded both the original repository and the checkout to version 1.8. (I dumped and imported the original repository and ran svn upgrade on the checkout.) The size of a checkout (excluding .svn) is 659 MB. The size of original repository dropped from 653 to 601 MB when I upgraded it to version 1.8 and now increased again to 738 MB due to new commits. But the weird thing is that the size of .svn folder in a local checkout is 2.7 GB. If I do a fresh checkout it's only 659M which is reasonable, but 2.7 GB seems too much. I then checked another checkout of a large repository. The size of the checkout is 6,3 GB, but the .svn folder apparently became a 36 GB monster. Is there any painless way to shrink the size of the local .svn folder (other than deleting everything and fetching the whole 6 GB again)? Thank you, Mojca
Re: Space optimisation of a local checkout
On Thu, Jan 16, 2014 at 3:03 PM, Stefan Sperling wrote: On Thu, Jan 16, 2014 at 02:49:56PM +0100, Mojca Miklavec wrote: Is there any painless way to shrink the size of the local .svn folder (other than deleting everything and fetching the whole 6 GB again)? 'svn cleanup' http://subversion.apache.org/docs/release-notes/1.7.html#wc-pristines Thank you very much for both the answer and the link (which includes the link to the tracker). Mojca
difference between svn copy -r REV SRC and SRC@REV
Hi, I was wondering whether -r REV SRC and SRC@REV in svn copy are supposed to behave differently or not. I was trying to recover a file that has already been deleted from the repository, but using -r syntax failed to work while @REV completed successfully: svn copy -r 10 ^/trunk/path/to/my/file.txt . svn: E160013: '/repository/name/!svn/bc/25/trunk/path/to/my/file.txt' path not found svn copy ^/trunk/path/to/my/file.txt@10 . A /here/file.txt On the other hand, if the file still exists, the -r REV syntax works just fine and copies the old file (revision 10 in the above example). Is this expected? I'm using SVN 1.8.5 (and also tried with the trunk version which behaves the same), while the server uses SVN 1.6.11. Mojca (PS: If necessary I can provide real URLs.)
Re: difference between svn copy -r REV SRC and SRC@REV
On Wed, Jan 15, 2014 at 10:19 PM, Ben Reser wrote: On 1/15/14, 1:05 PM, Mojca Miklavec wrote: I was wondering whether -r REV SRC and SRC@REV in svn copy are supposed to behave differently or not. Yes they have different behavior. You probably want to read about Peg and Operative Revisions here: http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html Thank you very much. I probably asked too fast. I just found the same link on http://stackoverflow.com/questions/6591938/svn-copy-r-revision-number-url-path (I was trying to find the information via svn help copy earlier, but the help page wasn't informative enough for this particular case.) Mojca
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Wed, Jan 8, 2014 at 4:53 PM, Philip Martin wrote: I get a problem with the checkout from your server using a trunk client. Very occasionally the checkout works but most of the time the client simply hangs while receiving the first file. It appears that the client is sending the REPORT request and receiving the response from the server. The client then pipelines 13 GET requests corresponding to the 13 files in the working copy. The server starts sending the response to the first GET and the client starts receiving it but the server never completes the response. The client hangs waiting for the server and eventually times out. If I use wireshark it shows the server sending an RST packet just before the client hangs. According to wireshark this is a Bad checksum packet. Wireshark shows the client retransmitting the GETs but there is no further server repsonse. I don't know enough to debug the problem further. I now upgraded the SVN client to 1.9.0-dev from trunk. With trunk version it's still inconsistent behaviour, but at least reproducible to a certain extent. I tried to checkout file a couple of times. Almost every time I get the following lines in error.log on the server: Unable to deliver content. [500, #0] Could not write data to filter. [500, #175002] but the first time the whole checkout finished successfully, even though the server first recorded 500 and apparently another 200 (success) on the second attempt for the same file. The client ended with success. The second time the client reported svn: E54: Error running context: Connection reset by peer (and the same happened when I ran it for the third/fourth/fifth/... time) Sometimes it works though. And it usually hangs on different files. I'm unable to reproduce the faulty behaviour if I do a checkout from the same network where the server is located, no matter what I try (upgrading SVN client doesn't help triggering the error). Philip also said that he had no problem doing a checkout with client version 1.8.5 or 1.7. With subversion client 1.8.5 I'm sometimes able to reproduce the problem from a different network, but it usually works. I tried wireshark, but I don't know what to do with the zillions of packets it shows me. I'll first try to copy the repository to another server to see if I could reproduce the problem from there. Other than that I would be grateful for any hints if there exists some painless way to debug the server. Mojca
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Wed, Jan 8, 2014 at 1:11 PM, Branko Čibej wrote: Hi Mojca, On 07.01.2014 20:58, Mojca Miklavec wrote: (The other problem with Error retrieving REPORT is still a mystery though.) Mojca I'm assuming your server is somewhere on the IJS network. Can you please ask the admins there if your server is behind a load balancer? I asked and it is not. Mojca
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Tue, Jan 7, 2014 at 12:41 PM, Philip Martin wrote: Mojca Miklavec writes: We have a server running Fedora which has recently been upgraded to version 20 and it's now running svn, version 1.8.5 (r1542147) I have a bunch of repositories served over http protocol with public read access and limited commit access. Shortly after the upgrade a weird behaviour has been noticed. Running svn up on the top level dir worked ok for me, but running svn co http://svn.myserver.net/myrepo/dirA fails with AdirA/subdir1 AdirA/subdir2 AdirA/subdir3 AdirA/subdir4 svn: E54: Error retrieving REPORT: Connection reset by peer The directory dirA contains one more file FILE.txt. Checking out any individual subdirN works and the browser is able to display the contents of dirA. Trying to click on FILE.txt in the browser sometimes works (it currently does) and sometimes shows an XML (like a few minutes ago, but I'm unable to get it now), saying something similar to the error I get in console***: svn: E175002: Unable to connect to a repository at URL 'svn.myserver.net/myrepo/dirA' svn: E175002: Unexpected HTTP status 500 'Internal Server Error' on '/myrepo/dirA' svn: E160004: Additional errors: svn: E160004: Corrupt node-revision '2-1.0.r137/330061' (*** To be precise: this is the error I get after upgrading the repository to the latest version of SVN, I didn't try to get to this error before upgrading.) The error.log in apache says just: [date] [dav:error] [pid 3613] [client ip:port] Unable to deliver content. [500, #0] [date] [dav:error] [pid 3613] [client ip:port] Could not write data to filter. [500, #175002] I first tried if upgrading the repository would help in any way, so I did svnadmin dump oldrepo | svnadmin load newrepo and checking the relevant revision r137 cited in the error all I see is the following (nothing unusual): --- Committed revision 136 Started new transaction, based on original revision 137 * editing path : dirA/FILE.txt ... done. * Dumped revision 137. * editing path : dirA/subdir1/somefile ... done. --- Committed revision 137 Checking out the same repository via http on the machine where the repository itself is located works fine. I'm using the same version of SVN (1.8.5) on Mac, but other svn clients on other OSes have problems as well. I tried checking the repository health with svnadmin verify /path/to/myrepo and all revisions passed except for some weird error inbetween (the file rev-prop-atomics.mutex is actually missing, but it isn't present in any other repository either): * Verifying repository metadata ... * Verifying metadata at revision 1 ... ... * Verifying metadata at revision 155 ... svnadmin: E160052: Revprop caching for '/path/to/myrepo/db' disabled because SHM infrastructure for revprop caching failed to initialize. svnadmin: E13: Can't open file '/path/to/myrepo/db/rev-prop-atomics.mutex': Permission denied * Verified revision 0. ... * Verified revision 160. I would appreciate any help or debugging hints. If necessary I can share the repository URL (but I would prefer to share it off-list to anyone interested in debugging). I can also try to debug myself, but I need some instructions telling me what to check. I didn't manage to find anything useful by googling the errors other than figuring out that the error was part of the code to fix a memory leak (http://svn.haxx.se/dev/archive-2009-08/0274.shtml). I've not seen E54 before but it is EXFULL which is some sort of network error. I suppose the corruption causes some sort of output problem. E13 is EACCES so you are running verify without write access to the repository. That seems like a perfectly reasonable thing to do so we should probably make the warning less intimidating. It's very odd that Apache is reporting corruption but both the dump/load and verify work without problem. Is the problem reproducible if you restart Apache? Yes, there is still a problem after restarting Apache. Even though it works for me at the moment and I tried fetching from multiple locations and servers, other users are still experiencing the same problem. Logs on the server confirm that. (Unable to deliver content. [500, #0] + Could not write data to filter. [500, #175002]) Mojca
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Tue, Jan 7, 2014 at 5:18 PM, Philip Martin wrote: Mojca Miklavec writes: Yes, there is still a problem after restarting Apache. Even though it works for me at the moment and I tried fetching from multiple locations and servers, other users are still experiencing the same problem. Logs on the server confirm that. (Unable to deliver content. [500, #0] + Could not write data to filter. [500, #175002]) Does the server log always contain the error: svn: E160004: Corrupt node-revision '2-1.0.r137/330061' I don't see that in the server log, but I was only checking error.log written by Apache server, I don't know where else to look, but I can check if you point me in the right direction. This error is sometimes displayed by the client (either in XML in the browser or as an error in the command line during svn up), but it's not consistent and it often works properly. It sometimes works in the first attempt, fails in the second one, and succeeds in the third attempt again. Only seconds or minutes apart. Is it always '2-1.0.r137/330061'? The exact revision reported as currupt depends on which subfolder I'm checking out. I believe it reports the last commit when files in that particular subfolder were modified. (I've seen this error when checking out two different subfolders. The number was always the same for the same subfolder, but different for different subfolders.) (It is a bit difficult to test because the behaviour is not consistent.) Mojca
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Tue, Jan 7, 2014 at 5:47 PM, Philip Martin wrote: Mojca Miklavec writes: On Tue, Jan 7, 2014 at 5:18 PM, Philip Martin wrote: Mojca Miklavec writes: Yes, there is still a problem after restarting Apache. Even though it works for me at the moment and I tried fetching from multiple locations and servers, other users are still experiencing the same problem. Logs on the server confirm that. (Unable to deliver content. [500, #0] + Could not write data to filter. [500, #175002]) Does the server log always contain the error: svn: E160004: Corrupt node-revision '2-1.0.r137/330061' I don't see that in the server log, but I was only checking error.log written by Apache server, I don't know where else to look, but I can check if you point me in the right direction. This error is sometimes displayed by the client (either in XML in the browser or as an error in the command line during svn up), but it's not consistent and it often works properly. It would be in the Apache error log. Ah, OK, I see it now in the old logs. There are no such lines in the latest logs. Are you saying that sometimes the client gets the E175002 error without the 'Corrupt node-revision' part? Yes. I'm attaching full log (with timestamps and IPs removed) for a certain period of time around 4th January. There are plenty of E175002 errors without any subsequent 'Corrupt node-revision' part, including all the latest entries (not part of the attachment). Are you saying that the client gets the 'Corrupt node-revision' error but it is not recorded in the error log? I was wrong about that. I was only checking the latest error log where all I get is [dav:error] [pid 42289] [IP:29011] Unable to deliver content. [500, #0] [dav:error] [pid 42289] [IP:29011] Could not write data to filter. [500, #175002] But I've found those additonal errors in an old (archived) log. At the moment I'm unable to reproduce the error 'Corrupt node-revision' both on the client and in server logs, but the repository is still misbehaving. It sometimes works in the first attempt, fails in the second one, and succeeds in the third attempt again. Only seconds or minutes apart. Is it always '2-1.0.r137/330061'? The exact revision reported as currupt depends on which subfolder I'm checking out. I believe it reports the last commit when files in that particular subfolder were modified. (I've seen this error when checking out two different subfolders. The number was always the same for the same subfolder, but different for different subfolders.) (It is a bit difficult to test because the behaviour is not consistent.) Which version of Apache are you using? Which Apache MPM are you using? Server version: Apache/2.4.7 (Unix) I'm not sure how to check MPM. I get httpd -l Compiled in modules: core.c mod_so.c http_core.c but httpd -V as suggested on some websites doesn't work. How should I check which MPM is being used? What sort of filesystem is used for the repository? Is it a local disk or a network disk? The repository is stored on a local disk. I'm not sure what about filesystem is it that you are asking, but here are some possibly relevant data: cat format 5 cat db/fs-type fsfs cat db/format 6 layout sharded 1000 (and before I upgraded the repository, db/format was 4). Is that what you were asking or do did you want to know something else? Mojca error.log Description: Binary data
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Tue, Jan 7, 2014 at 7:34 PM, Philip Martin wrote: So you used dump/load to create a new repository and then replaced the old repository with the new repository? If you did that while Apache was running, without restarting Apache, then that explains the 'Corrupt node-revision' error as you changed the data on disk. Ah, thanks a lot for explaining that. Yes, I did dump/load the old repository into a new one because I wanted to test if it would solve the problem (on client) svn: E54: Error retrieving REPORT: Connection reset by peer (on server) [dav:error] [pid 3613] [client ip] Unable to deliver content. [500, #0] [dav:error] [pid 3613] [client ip] Could not write data to filter. [500, #175002] which it didn't. It only added a few additional problems until I restarted Apache (I'm sorry for confusing you with those), but the initial error E54/175002 is still causing problems. What you are left with is some sort of intermittent network problem. I don't know what is causing that. Is there any way to debug that? Thank you very much, Mojca
Re: Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
On Tue, Jan 7, 2014 at 8:44 PM, Ryan Schmidt wrote: On Jan 7, 2014, at 12:01, Mojca Miklavec wrote: On Tue, Jan 7, 2014 at 5:47 PM, Philip Martin wrote: Which version of Apache are you using? Which Apache MPM are you using? Server version: Apache/2.4.7 (Unix) I'm not sure how to check MPM. I get httpd -l Compiled in modules: core.c mod_so.c http_core.c but httpd -V as suggested on some websites doesn't work. How should I check which MPM is being used? In what way does “httpd -V” not work? In this way: httpd -V [time] [so:warn] [pid 63924] AH01574: module dav_svn_module is already loaded, skipping [time] [so:warn] [pid 63924] AH01574: module authz_svn_module is already loaded, skipping AH00548: NameVirtualHost has no effect and will be removed in the next release /path/to/00-vhosts.conf:1 (13)Permission denied: AH02291: Cannot access directory '/path/to/logs/1/' for error log of vhost defined at /path/to/20-another.conf:4 ... ... (repeats a bunch of times) ... AH00014: Configuration check failed But I saw the trick now. It wants me to use sudo httpd -V for some reason, then it works. And yes, it's prefork in my case as well, but that's probably no longer relevant now that one mystery with forgotten Apache restart was solved. (The other problem with Error retrieving REPORT is still a mystery though.) Mojca
Seeking help for E000054: Error retrieving REPORT: Connection reset by peer
Hello, We have a server running Fedora which has recently been upgraded to version 20 and it's now running svn, version 1.8.5 (r1542147) I have a bunch of repositories served over http protocol with public read access and limited commit access. Shortly after the upgrade a weird behaviour has been noticed. Running svn up on the top level dir worked ok for me, but running svn co http://svn.myserver.net/myrepo/dirA fails with AdirA/subdir1 AdirA/subdir2 AdirA/subdir3 AdirA/subdir4 svn: E54: Error retrieving REPORT: Connection reset by peer The directory dirA contains one more file FILE.txt. Checking out any individual subdirN works and the browser is able to display the contents of dirA. Trying to click on FILE.txt in the browser sometimes works (it currently does) and sometimes shows an XML (like a few minutes ago, but I'm unable to get it now), saying something similar to the error I get in console***: svn: E175002: Unable to connect to a repository at URL 'svn.myserver.net/myrepo/dirA' svn: E175002: Unexpected HTTP status 500 'Internal Server Error' on '/myrepo/dirA' svn: E160004: Additional errors: svn: E160004: Corrupt node-revision '2-1.0.r137/330061' (*** To be precise: this is the error I get after upgrading the repository to the latest version of SVN, I didn't try to get to this error before upgrading.) The error.log in apache says just: [date] [dav:error] [pid 3613] [client ip:port] Unable to deliver content. [500, #0] [date] [dav:error] [pid 3613] [client ip:port] Could not write data to filter. [500, #175002] I first tried if upgrading the repository would help in any way, so I did svnadmin dump oldrepo | svnadmin load newrepo and checking the relevant revision r137 cited in the error all I see is the following (nothing unusual): --- Committed revision 136 Started new transaction, based on original revision 137 * editing path : dirA/FILE.txt ... done. * Dumped revision 137. * editing path : dirA/subdir1/somefile ... done. --- Committed revision 137 Checking out the same repository via http on the machine where the repository itself is located works fine. I'm using the same version of SVN (1.8.5) on Mac, but other svn clients on other OSes have problems as well. I tried checking the repository health with svnadmin verify /path/to/myrepo and all revisions passed except for some weird error inbetween (the file rev-prop-atomics.mutex is actually missing, but it isn't present in any other repository either): * Verifying repository metadata ... * Verifying metadata at revision 1 ... ... * Verifying metadata at revision 155 ... svnadmin: E160052: Revprop caching for '/path/to/myrepo/db' disabled because SHM infrastructure for revprop caching failed to initialize. svnadmin: E13: Can't open file '/path/to/myrepo/db/rev-prop-atomics.mutex': Permission denied * Verified revision 0. ... * Verified revision 160. I would appreciate any help or debugging hints. If necessary I can share the repository URL (but I would prefer to share it off-list to anyone interested in debugging). I can also try to debug myself, but I need some instructions telling me what to check. I didn't manage to find anything useful by googling the errors other than figuring out that the error was part of the code to fix a memory leak (http://svn.haxx.se/dev/archive-2009-08/0274.shtml). Thank you, Mojca
Re: how to repeatedly checkout sparse directories in a cron job
On Wed, Dec 21, 2011 at 13:21, Stefan Sperling wrote: On Wed, Dec 21, 2011 at 06:07:05AM -0600, Ryan Schmidt wrote: On Dec 20, 2011, at 06:08, Mojca Miklavec wrote: Is there any way to prevent deleting existing contents within the scope of command line arguments? A workaround is to use something like if [ ! -d project ]; then svn co --depth=empty $URL fi but I would be really happy if there was some command like: please checkout $URL, but no need to fetch any files yet, in particular don't fetch 'branches'; on the other hand please don't delete 'trunk' and 'tags' if already present I don't think that exists as a built-in command as such. But probably playing with the depth settings you can get what you want. I am not very familiar with these settings however so I cannot advise. svn-viewspec.py might help: https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svn-viewspec.py Thank you very much for pointing out this script. I think that it will perfectly suit the need. Mojca
Re: E175002: The POST request returned invalid XML in the response: XML parse error
On Tue, Dec 20, 2011 at 01:43, Daniel Shahaf wrote: Mojca Miklavec wrote on Mon, Dec 19, 2011 at 16:56:05 +0100: May I ask for any hints of what could go wrong and how I could debug the problem? I have seen various threads after searching with google, but none of those that I checked provided any useful insight. (I work with SVN a lot, but have never experienced a problem like this one.) The use of POST request is new in 1.7 (client+server must be 1.7 for this). They are. Can you capture the XML response on the wire and share it here? (with wireshark, or neon-debug-mask) If it contains sensitive stuff feel free to share it by private mail. Maybe Philip Martin's mail already answers this question. I first need to figure out how to use wireshark. Is there any chapter in SVN documentation explaining how to troubleshoot problems with subversion on a network level? And if not - would it be feasible to prepare one? Mojca
Re: E175002: The POST request returned invalid XML in the response: XML parse error
On Tue, Dec 20, 2011 at 02:34, Philip Martin wrote: Mojca Miklavec writes: svn ci -m some comment svn: E175002: Commit failed (details follow): svn: E175002: The POST request returned invalid XML in the response: XML parse error at line 3: not well-formed (invalid token) (/suite/!svn/me) One thing that can cause this error is when the OS permissions prevent Apache writing to the repository. Thank you very much for the hint. This is a quite likely cause. I don't have the computer at hand, but I indeed created the repository with sudo svncreate and if apache needs to write to it, I can easily believe that the rights are not sufficient to do so. Still, it would help a lot if the error message was more descriptive than it was (I would then figure out what to do myself). The underlying error on the server is: Can't open file '.../db/txn-current-lock': Permission denied but it looks like we have a problem with the error reporting. I don't see anything logged in the apache log, and wireshark shows the response HTTP/1.1 500 Internal Server Error Date: Tue, 20 Dec 2011 01:22:45 GMT Server: Apache/2.2.16 (Debian) mod_ssl/2.2.16 OpenSSL/0.9.8o DAV/2 SVN/1.8.0-dev Content-Length: 207 Connection: close Content-Type: text/xml; charset=utf-8 ?xml version=1.0 encoding=utf-8? D:error xmlns:D=DAV: xmlns:m=http://apache.org/dav/xmlns; D:(null)/ m:human-readable errcode=13 could not begin a transaction /m:human-readable /D:error That D:(null) is suspicious. Thank you for the precise analysis. Mojca
update --depth=empty ignored since 1.7
Dear list, I'm experiencing some weird behaviour with SVN 1.7. The following sequence of commands fails to respect --depth=empty: svn co --depth=empty http://foundry.supelec.fr/svn/metapost svn up --depth=empty metapost/tags (you can use 'anonymous' as a username) The first command runs fine, but the second one starts fetching all tags. If I try to open repository in web browser, it reports that the server is using SVN 1.4.2 (r22196) which is a bit old, but with SVN 1.6.16 the same sequence of commands worked fine - depth=empty seems to be respected. I cannot reproduce the behaviour with http://svn.apache.org/repos/asf/subversion where the same commands work just fine. Is this a bug in SVN client 1.7.(0/1) or my misunderstanding of how the above commands are supposed to work? I noticed that some depth=empty-related issues are mentioned in changelog for 1.7.1, but subversion 1.7.1 as shipped with MacPorts on Mac OS X 10.7 doesn't seem to behave any differently. Thank you very much, Mojca
update --depth=empty ignored since 1.7
Dear list, I'm experiencing some weird behaviour of SVN 1.7. The following sequence of commands fails to respect --depth=empty: svn co --depth=empty http://foundry.supelec.fr/svn/metapost svn up --depth=empty metapost/tags (you can use 'anonymous' as a username if it asks) The first command runs fine, but the second one starts fetching all tags. The workaround is to use svn up --set-depth empty metapost/tags before fetching anything with svn up, but --depts=empty seems to be respected in 1.6.16 and also makes a lot more sense the way it worked earlier. Is this a bug or my misunderstanding of how the above commands are supposed to work? I noticed that some depth=empty-related issues are mentioned in changelog for 1.7.1, but subversion 1.7.1 as shipped with MacPorts on Mac OS X 10.7 doesn't seem to solve the problem for me. Thank you very much, Mojca
