Re: svn 1.3 crashing after sometime
Backup your repository *FIRST*. On Sun, Jul 4, 2010 at 11:11 AM, west alto westa...@gmail.com wrote: Thanks guys, i'm new to upgrade using your way. which is safer using svnadmin dump/load or svnadmin upgrade/verify or svnadmin hotcopy given my situation where users can afford downtime. Please check if my process is correct: - remove svn 1.3 and all its dependencies. - install svn 1.6 - backup my repository - svnadmin upgrade /path/to/repository - svnadmin verify /path/to/repository After the steps above, is my repository now usable for 1.6? any other steps so i could tweak my repository for best performance? or without going svnadmin upgrade and verify, i just run svnadmin hotcopy /path/to/old_repo_svn_1.3 /path/to/new_repo gives me an error. how can i handle this safely. svn01:/srv/svn # svnadmin hotcopy /srv/svn/test_repos /srv/svn/new_repos svnadmin: Can't open file '/srv/svn/test_repos/db/fsfs.conf': No such file or directory Thanks guys for your help. West On Sun, Jul 4, 2010 at 2:11 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Ryan Schmidt wrote on Sat, 3 Jul 2010 at 20:26 -: what difference and advantage bet. linear and sharded? It's explained here, a bit: http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.tk.fsfsreshard (You don't need the fsfs-reshard script mentioned there; that was just the only place I found a description of a sharded repository in my quick search.) The canonical definition is in the Filesystem format options section of http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure. Daniel (don't edit the 'format' file by hand, though)
Re: svn 1.3 crashing after sometime
Hello, Your repository layout is probably linear now. According to the number of revisions, revision sizes and the underlying operating system it may be interesting to switch to sharded layout and tune the shard size (default is 1000 revisions): http://ymartin59.free.fr/wordpress/index.php/projets/tune-subversion-fsfs-repository/ A good performance test case is to measure time required by svnadmin verify to check your repository. Best regards, - west alto westa...@gmail.com a écrit : Thanks guys, i'm new to upgrade using your way. which is safer using svnadmin dump/load or svnadmin upgrade/verify or svnadmin hotcopy given my situation where users can afford downtime. Please check if my process is correct: - remove svn 1.3 and all its dependencies. - install svn 1.6 - backup my repository - svnadmin upgrade /path/to/repository - svnadmin verify /path/to/repository After the steps above, is my repository now usable for 1.6? any other steps so i could tweak my repository for best performance? -- Yves Martin
Re: svn 1.3 crashing after sometime
running svnadmin hotcopy does not touch the source repository? what difference and advantage bet. linear and sharded? Thanks, West On Fri, Jul 2, 2010 at 8:12 PM, Nico Kadel-Garcia nka...@gmail.com wrote: On Fri, Jul 2, 2010 at 3:18 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Nico Kadel-Garcia wrote on Thu, 1 Jul 2010 at 23:11 -0400: On Thu, Jul 1, 2010 at 10:04 PM, west alto westa...@gmail.com wrote: Thanks guys, Ho do i upgrade from 1.3 to 1.6? Do i need to install first 1.4 then do a svn dump and load, then 1.5 and then svn dump and load etc. No, move aside the repository, do a good backup to tape, then install version 1.6. Then do an svnadmin hotcopy with version 1.6, which should build you a nice clean upgraded I have tested it and I think you are mistaken here: running 'svnadmin1.7 hotcopy' on a 1.4-format FSFS repository produces another 1.4-format FSFS repository. Oh. I see that you're right, and I'd left out the svnadmin upgrade the-new-repo step. [[[ % rm -rf t % /usr/bin/svnadmin create t --pre-1.5-compatible % cat t/**/format 2 5 % svntrunk % svnadmin hotcopy t t2 % cat t2/**/format 2 5 % svnadmin --version svnadmin, version 1.7.0 (dev build)? compiled Jul 1 2010, 12:12:57 ]]] I suggest a dump|load instead. Dumpload is, historically, rather fragile and doesn't bring over your configuration files. This is particularly critical if you have modified hook scripts and configuration files. Resolving those can be an adventure.
Re: svn 1.3 crashing after sometime
On Sat, Jul 3, 2010 at 12:03 PM, west alto westa...@gmail.com wrote: running svnadmin hotcopy does not touch the source repository? what difference and advantage bet. linear and sharded? Thanks, It builds the new repo, does a dump and reload (in the same format, which I hadn'e realized), then copies over the configuration files. The advantage is that your original repository stays untouched, so minor changes in the dump processing (such as the new version 1.6 handling of logs to remove extraneous EOL characters and improved non-Ascii character handling) can be applied in the process.
Re: svn 1.3 crashing after sometime
On Jul 3, 2010, at 11:03, west alto wrote: running svnadmin hotcopy does not touch the source repository? Correct. what difference and advantage bet. linear and sharded? It's explained here, a bit: http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.tk.fsfsreshard (You don't need the fsfs-reshard script mentioned there; that was just the only place I found a description of a sharded repository in my quick search.)
Re: svn 1.3 crashing after sometime
Ryan Schmidt wrote on Sat, 3 Jul 2010 at 20:26 -: what difference and advantage bet. linear and sharded? It's explained here, a bit: http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.tk.fsfsreshard (You don't need the fsfs-reshard script mentioned there; that was just the only place I found a description of a sharded repository in my quick search.) The canonical definition is in the Filesystem format options section of http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure. Daniel (don't edit the 'format' file by hand, though)
Re: svn 1.3 crashing after sometime
Nico Kadel-Garcia wrote on Thu, 1 Jul 2010 at 23:11 -0400: On Thu, Jul 1, 2010 at 10:04 PM, west alto westa...@gmail.com wrote: Thanks guys, Ho do i upgrade from 1.3 to 1.6? Do i need to install first 1.4 then do a svn dump and load, then 1.5 and then svn dump and load etc. No, move aside the repository, do a good backup to tape, then install version 1.6. Then do an svnadmin hotcopy with version 1.6, which should build you a nice clean upgraded I have tested it and I think you are mistaken here: running 'svnadmin1.7 hotcopy' on a 1.4-format FSFS repository produces another 1.4-format FSFS repository. [[[ % rm -rf t % /usr/bin/svnadmin create t --pre-1.5-compatible % cat t/**/format 2 5 % svntrunk % svnadmin hotcopy t t2 % cat t2/**/format 2 5 % svnadmin --version svnadmin, version 1.7.0 (dev build) compiled Jul 1 2010, 12:12:57 ]]] I suggest a dump|load instead. and compatible copy you should put in place of the old repository.
Re: svn 1.3 crashing after sometime
Hello, For an upgrade from 1.3 to 1.6, I have simply apply svnadmin upgrade command and check integrity with svnadmin verify. Then, the repository is still in linear mode - which is not an issue, it works. But for performance reason, it is better to move to sharded mode. To do so, I have worked on fsfs-reshard.py script. You will find it with documentation how to use it safely here: http://ymartin59.free.fr/wordpress/index.php/tune-subversion-fsfs-repository/ Kind regards - Nico Kadel-Garcia nka...@gmail.com a écrit : On Thu, Jul 1, 2010 at 10:04 PM, west alto westa...@gmail.com wrote: Thanks guys, Ho do i upgrade from 1.3 to 1.6? Do i need to install first 1.4 then do a svn dump and load, then 1.5 and then svn dump and load etc. No, move aside the repository, do a good backup to tape, then install version 1.6. Then do an svnadmin hotcopy with version 1.6, which should build you a nice clean upgraded and compatible copy you should put in place of the old repository. -- Yves Martin
Re: svn 1.3 crashing after sometime
Sorry, page link has changed: http://ymartin59.free.fr/wordpress/index.php/projets/tune-subversion-fsfs-repository/ Comments and feedback are welcome - Yves Martin ymarti...@free.fr a écrit : Hello, For an upgrade from 1.3 to 1.6, I have simply apply svnadmin upgrade command and check integrity with svnadmin verify. Then, the repository is still in linear mode - which is not an issue, it works. But for performance reason, it is better to move to sharded mode. To do so, I have worked on fsfs-reshard.py script. You will find it with documentation how to use it safely here: http://ymartin59.free.fr/wordpress/index.php/tune-subversion-fsfs-repository/ Kind regards -- Yves Martin
Re: svn 1.3 crashing after sometime
On Fri, Jul 2, 2010 at 3:18 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Nico Kadel-Garcia wrote on Thu, 1 Jul 2010 at 23:11 -0400: On Thu, Jul 1, 2010 at 10:04 PM, west alto westa...@gmail.com wrote: Thanks guys, Ho do i upgrade from 1.3 to 1.6? Do i need to install first 1.4 then do a svn dump and load, then 1.5 and then svn dump and load etc. No, move aside the repository, do a good backup to tape, then install version 1.6. Then do an svnadmin hotcopy with version 1.6, which should build you a nice clean upgraded I have tested it and I think you are mistaken here: running 'svnadmin1.7 hotcopy' on a 1.4-format FSFS repository produces another 1.4-format FSFS repository. Oh. I see that you're right, and I'd left out the svnadmin upgrade the-new-repo step. [[[ % rm -rf t % /usr/bin/svnadmin create t --pre-1.5-compatible % cat t/**/format 2 5 % svntrunk % svnadmin hotcopy t t2 % cat t2/**/format 2 5 % svnadmin --version svnadmin, version 1.7.0 (dev build) compiled Jul 1 2010, 12:12:57 ]]] I suggest a dump|load instead. Dumpload is, historically, rather fragile and doesn't bring over your configuration files. This is particularly critical if you have modified hook scripts and configuration files. Resolving those can be an adventure.
Re: svn 1.3 crashing after sometime
First, upgrade from version 1.3. I recognize that professional grade Linux distributions consider software stability critical, but Subversion is still significantly evolving, and the number of important upgrades since version 1.3 is quite large. In particular, working copies that are checked out with tools like contemporary TortoiseSVN are 1.6 based, and are not manageable with Subversion 1.3 command line tools. Second: throw HTTPS based access the heck out. It has always presented a security issue because the UNIX and Linux Subversion client stores passwords in clear text, no matter which protocol you use. Use ssh+svn if you need protected access, with SSH keys managed to a specific Subversion user account and the stored public keys tweaked to register specific usernames. That will probably leapfog you right past this problem. On Wed, Jun 30, 2010 at 10:42 PM, west alto westa...@gmail.com wrote: Hi Gurus, Kindly help me with my problem. Subversion is running fine for sometime and then system load suddenly goes high and no one can login to the server. 172.23.14.1 - - [01/Jul/2010:10:18:36 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:36 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:36 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:37 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:37 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:37 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 Here my spec: svn 1.3 apache 2.2 sles10 sp3 authenticates with windows ad - clients are mostly tortoise (1.5 to 1.6) but there use are very basic: checkin, checkout, copy, mv, update - 90 repositories hosted - one hudson client running every 30 mins which connects to 50+ repositories every repository are defined this way in apache: Location /abc DAV svn SVNPath /srv/svn/abc AuthName Please use your ACTIVE DIRECTORY for Authentication AuthType Basic AuthBasicProvider ldap AuthzLDAPAuthoritative off Include /etc/apache2/.ldapbinddn AuthLDAPURL ldaps://192.168.1.1 192.168.1.2:636/OU=ADBC,DC=def,DC=local?sAMAccountName?sub?(objectClass=user) SSLRequireSSL AuthzSVNAccessFile /etc/apache2/svn_acl/abc Require valid-user SVNPathAuthz off /Location mod_ldap setting: LDAPTrustedMode SSL LDAPVerifyServerCert off LDAPSharedCacheSize 50 LDAPCacheEntries 1024 LDAPCacheTTL 43200 LDAPOpCacheEntries 1024 LDAPOpCacheTTL 43200 LDAPConnectionTimeout 3 sysctl.conf: net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.ip_forward = 1 net.ipv4.tcp_syncookies = 1 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_timestamps = 1 net.ipv4.tcp_sack = 1 net.ipv4.tcp_no_metrics_save = 1 net.ipv4.tcp_moderate_rcvbuf = 1 net.core.netdev_max_backlog = 2500 I saw this in my logs: [Wed Jun 30 18:14:13 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 18:14:23 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 18:56:17 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 18:56:22 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:01:07 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:01:10 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:06:43 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:06:48 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:12:15 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:12:25 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:12:37 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:13:12 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:15:38 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:18:24 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:18:44 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:20:27 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] Thanks,
Re: svn 1.3 crashing after sometime
On Thu, Jul 1, 2010 at 10:04 PM, west alto westa...@gmail.com wrote: Thanks guys, Ho do i upgrade from 1.3 to 1.6? Do i need to install first 1.4 then do a svn dump and load, then 1.5 and then svn dump and load etc. No, move aside the repository, do a good backup to tape, then install version 1.6. Then do an svnadmin hotcopy with version 1.6, which should build you a nice clean upgraded and compatible copy you should put in place of the old repository.
svn 1.3 crashing after sometime
Hi Gurus, Kindly help me with my problem. Subversion is running fine for sometime and then system load suddenly goes high and no one can login to the server. 172.23.14.1 - - [01/Jul/2010:10:18:36 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:36 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:36 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:37 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:37 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 172.23.14.1 - - [01/Jul/2010:10:18:37 +0800] PROPFIND /xyz/branches/prod HTTP/1.1 401 1263 Here my spec: svn 1.3 apache 2.2 sles10 sp3 authenticates with windows ad - clients are mostly tortoise (1.5 to 1.6) but there use are very basic: checkin, checkout, copy, mv, update - 90 repositories hosted - one hudson client running every 30 mins which connects to 50+ repositories every repository are defined this way in apache: Location /abc DAV svn SVNPath /srv/svn/abc AuthName Please use your ACTIVE DIRECTORY for Authentication AuthType Basic AuthBasicProvider ldap AuthzLDAPAuthoritative off Include /etc/apache2/.ldapbinddn AuthLDAPURL ldaps://192.168.1.1 192.168.1.2:636/OU=ADBC,DC=def,DC=local?sAMAccountName?sub?(objectClass=user) SSLRequireSSL AuthzSVNAccessFile /etc/apache2/svn_acl/abc Require valid-user SVNPathAuthz off /Location mod_ldap setting: LDAPTrustedMode SSL LDAPVerifyServerCert off LDAPSharedCacheSize 50 LDAPCacheEntries 1024 LDAPCacheTTL 43200 LDAPOpCacheEntries 1024 LDAPOpCacheTTL 43200 LDAPConnectionTimeout 3 sysctl.conf: net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.ip_forward = 1 net.ipv4.tcp_syncookies = 1 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_timestamps = 1 net.ipv4.tcp_sack = 1 net.ipv4.tcp_no_metrics_save = 1 net.ipv4.tcp_moderate_rcvbuf = 1 net.core.netdev_max_backlog = 2500 I saw this in my logs: [Wed Jun 30 18:14:13 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 18:14:23 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 18:56:17 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 18:56:22 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:01:07 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:01:10 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:06:43 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:06:48 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:12:15 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:12:25 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:12:37 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:13:12 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:15:38 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:18:24 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:18:44 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] [Wed Jun 30 19:20:27 2010] [error] [client 172.23.139.251] The requested report is unknown. [501, #27] Thanks, West
