Re: difference between 1.8.1x and 1.9.9 in UUID creation/handling
On 2017-02-15 23:02, Stefan wrote: > Hi Olli, > On 2/15/2017 22:32, olli hauer wrote: >> Hi, >> >> I just noticed a UUID difference between svnadmin 1.8.15 and 1.9.5, but >> haven't found the documentation for this behavior >> Is it normal to have two UUID's per default if the repo is created with >> svnadmin 1.9.x? >> >> >> Test with svn 1.8 >> $ svnadmin create new_repo >> $ svnadmin dump old_repo/ | svnadmin dump new_repo/ >> $ cmd old_repo/db/uuid new_repo/db/uuid ; echo $? >> 0 >> >> >> Test with svn 1.9.4/1.9.5 >> $ svn info file:///path/to/old_repo | grep UUID >> Repository UUID: 228ca703-1480-e111-95e2-001999006c86 >> >> $ cat old_repo/db/uuid >> 228ca703-1480-e111-95e2-001999006c86 >> >> $ svnadmin create new_repo >> $ cat new_repo/db/uuid >> ee6bf0e3-c2f3-e611-bd38-000c2934dcba >> 936cf0e3-c2f3-e611-bd38-000c2934dcba <-- there is a second UUID >> >> >> $ svn info file:///path/to/new_repo | grep UUID >> Repository UUID: ee6bf0e3-c2f3-e611-bd38-000c2934dcba >> >> >> Dump / Load into new_repo >> $ svnadmin dump old_repo/ | svnadmin dump new_repo/ >> >> $ cat new_repo/db/uuid >> 228ca703-1480-e111-95e2-001999006c86<-- the old repo UUID >> e2e3759b-c3f3-e611-bd38-000c2934dcba<-- the old (first) UUID before the >> load cycle >> >> $ svn info file:///path/to/new_repo | grep UUID >> Repository UUID: 228ca703-1480-e111-95e2-001999006c86 > have a look at this thread. This should explain the principles a bit: > http://mail-archives.apache.org/mod_mbox/subversion-dev/201702.mbox/%3C95a2cd32-c45a-f997-e51d-a4d86e9902f5%40apache.org%3E > > Note that the second line in the UUID file is not the "real" UUID > (that's the first one) and it's related to the fsfs-format whether you > have a second line or not (it got introduced in fsfs format 7 which is > the default fsfs-format when you create a repository with SVN 1.9). > > Regards, > Stefan > Hi Stefan, thanks that was the missing part I haven't found in the release notes -- Regards, olli
difference between 1.8.1x and 1.9.9 in UUID creation/handling
Hi, I just noticed a UUID difference between svnadmin 1.8.15 and 1.9.5, but haven't found the documentation for this behavior Is it normal to have two UUID's per default if the repo is created with svnadmin 1.9.x? Test with svn 1.8 $ svnadmin create new_repo $ svnadmin dump old_repo/ | svnadmin dump new_repo/ $ cmd old_repo/db/uuid new_repo/db/uuid ; echo $? 0 Test with svn 1.9.4/1.9.5 $ svn info file:///path/to/old_repo | grep UUID Repository UUID: 228ca703-1480-e111-95e2-001999006c86 $ cat old_repo/db/uuid 228ca703-1480-e111-95e2-001999006c86 $ svnadmin create new_repo $ cat new_repo/db/uuid ee6bf0e3-c2f3-e611-bd38-000c2934dcba 936cf0e3-c2f3-e611-bd38-000c2934dcba <-- there is a second UUID $ svn info file:///path/to/new_repo | grep UUID Repository UUID: ee6bf0e3-c2f3-e611-bd38-000c2934dcba Dump / Load into new_repo $ svnadmin dump old_repo/ | svnadmin dump new_repo/ $ cat new_repo/db/uuid 228ca703-1480-e111-95e2-001999006c86<-- the old repo UUID e2e3759b-c3f3-e611-bd38-000c2934dcba<-- the old (first) UUID before the load cycle $ svn info file:///path/to/new_repo | grep UUID Repository UUID: 228ca703-1480-e111-95e2-001999006c86 -- Regards, olli
svnsync: Assertion failed: (svn_fspath__is_canonical(child_fspath)), ...
I'm running into a bug in svnsync that was fixed some years ago for svn http://subversion.tigris.org/issues/show_bug.cgi?id=4088 The FreeBSD project is now running for users subversion mirror without svn:// access. The main issue I'm running into is a bug because the repos are located on /$repo and not on a sub path like /svn/$repo Running my existing svnsync repo against the new URL results in a core dump of svnsync Assertion failed: (svn_fspath__is_canonical(child_fspath)), function svn_fspath__skip_ancestor, file subversion/libsvn_subr/dirent_uri.c, line 2492. Abort trap (core dumped) Until now I have to clean after each svnsync the svnsync lock with $ svn pd svn:sync-lock --revprop -r 0 file:///$repo Details if from interest: svn properties (r0): svn:date : 2012-07-14T06:36:59.701452Z svn:sync-from-url: https://svn.freebsd.org/ports/ svn:sync-from-uuid : 35697150-7ecd-e111-bb59-0022644237b5 svn:sync-last-merged-rev : 392393 $ svn --version svn, version 1.8.13 (r1667537) compiled Jul 2 2015, 04:04:19 on amd64-portbld-freebsd10.1 ... * ra_svn : Module for accessing a repository using the svn network protocol. - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - using serf 1.3.8 - handles 'http' scheme - handles 'https' scheme Should I re- open bug #4088 ? -- Regards, olli
Re: SVN skipping revert after add for file path with spaces in it
On 2014-09-05 22:37, Simone Agha wrote: Hi, I'm having trouble un-adding a directory (recursively) that has spaces in its name. I am doing this on Windows 8.1 through a batch script. An example of my code is below: set PATHWSPACES=%CD% svn add PATHWSPACES\subdir1\ svn revert PATHWSPACES\subdir1\subdir2\ --recursive When I run the batch script, the files are added correctly, but I get this line when the directory is supposed to be reverted: Skipped 'C:\Path with spaces\subdir1\subdir2' Setting the PATHWSPACES variable using quotes of any kind does not work. However, what does work is typing out the complete path which is not feasible for what I'm trying to do with the batch script. So, this works: Set PATHWSPACES=%CD% svn add PATHWSPACES\subdir1\ svn revert C:\Path with spaces\subdir1\subdir2\ --recursive Putting quotes around the entire path or putting quotes around the variable and the entire path also does not work. I get the following messages: For code that is: svn revert PATHWSPACES\subdir1\subdir2 --recursive svn: E155007: 'C:Path' is not a working copy For code that is: svn revert PATHWSPACES\subdir1\subdir2\ --recursive Skipped 'C:\Path with spaces\subdir1\subdir2 -recursive' Help! What is going on? Unfortunately, I can't change to a path name without spaces. I can think about two solutions 1) instead using PATHWSPACES\subdir1\ use PATHWSPACES\subdir1\ 2) cd PATHWSPACES\subdir1\ svn revert -R Even windows self has today issues with masking spaces in the path, you will find x samples with quoted paths in the registry or on technet. Not exactly your issue but describes how to use cmd and quotes http://technet.microsoft.com/en-us/library/cc771320.aspx
dav issue with subversion-1.7.18 and httpd-2.4.10
Upgraded a subversion-1.7.18 installation from http-2.2.28 to httpd-2.4.10 and running into strange issues. Two clean builds (jail environment) with apr-1.5.1, apr-util-1.5.3, subversion-1.7.18, a) httpd-2.2.28 and b) httpd-2.4.10 No issues on httpd-2.2.28 or by using the svn protocol only via webdav (client independ) Form the http-2.4.10 error log: [Tue Aug 12 08:46:55.575078 2014] [dav:error] [pid 96427] [client 1.2.3.5:57266] Unable to PUT new contents for /svn/repo_abc/!svn/wrk/d6caf7c8-4701-0010-b3a9-ef26e25e095f/branches/Release_2014_08/file.name. [403, #0] [Tue Aug 12 08:46:55.575126 2014] [dav:error] [pid 96427] [client 1.2.3.5:57266] Could not prepare to write the file [500, #200014] [Tue Aug 12 08:46:55.575137 2014] [dav:error] [pid 96427] [client 1.2.3.5:57266] Base checksum mismatch on '/branches/Release_2014_08/file.name':\n expected: e02e545a6b7d4f347d6e8964db65e51e\n actual: 108b3c0ab68c457d8da178558187c3f9\n [500, #200014] [Tue Aug 12 08:56:26.428469 2014] [dav:error] [pid 96427] [client 1.2.3.6:50011] Provider encountered an error while streaming a REPORT response. [500, #0] [Tue Aug 12 08:56:26.428535 2014] [dav:error] [pid 96427] (32)Broken pipe: [client 1.2.3.6:50011] Error flushing brigade. [500, #0] - switching from http(s)// to svn:// and the issue is no longer present. - rebuild everything against http-2.2.27 everything is fine again. - testing the repo with `svnadmin verify' shows no errors. The issue is also present after a fresh checkout and modify one file, regardless the used client (1.7/1.8, native client, tortoise, svnkit, javahl) Any hint where to start searching for the issue (latest httpd mod_dav changes or subversion missing backports)? -- Regards, olli
Re: dav issue with subversion-1.7.18 and httpd-2.4.10
On 2014-08-13 23:06, Mark Phippard wrote: On Wed, Aug 13, 2014 at 5:00 PM, olli hauer oha...@gmx.de wrote: Upgraded a subversion-1.7.18 installation from http-2.2.28 to httpd-2.4.10 and running into strange issues. Two clean builds (jail environment) with apr-1.5.1, apr-util-1.5.3, subversion-1.7.18, a) httpd-2.2.28 and b) httpd-2.4.10 No issues on httpd-2.2.28 or by using the svn protocol only via webdav (client independ) Not to hijack the thread off topic, but I am sure you are aware that httpd 2.2.x and 2.4.x are not binary compatible. So you will need a different set of SVN binaries built for each version? Or, if the APR etc. libs are all the same, then at least mod_dav_svn would need to be custom built for each. Sure, therefore I setup two builds, one against a) httpd-2.2.28 and the second against b) httpd-2.4.10. Package building is one part of my work ;) Removing all related packages changing the package repo and reinstall is done in one minute. Did you consider trying 1.8.x? I recall there were some 2.4.x fixes made. Yes, but unluckily 1.8 is no option at the moment, also this could be a potential problem for RHEL7 based systems (they deliver with svn 1.7 and httpd 2.4) I assume those were backported to 1.7.x but do not remember. Reading the Changelog it seems so, but i suspect some changes are missing after the latest changes on the httpd-2.4 dav changes.
Re: Can't move temporary on update.
On 2014-03-02 11:40, Stefan wrote: Hi, I came up with an even easier repro case. It seems to suffice to trigger the problem by simply committing the problematic file to an empty local repository and having avast installed. To whom should I send the repro-case (it requires the zipped-exe-file). Hi Stefan, if it works without installed avast or with a different virus scanner installed, then I suggest to open a ticket by avast. sry for the belated answer. It took me a while before I had some time to investigate that issue deeper. Following investigations were done with TSVN 1.8.5 (Subversion 1.8.8) and Server is now running Visual SVN 2.7.3 (based on SVN 1.8.5). I traced the issue down to obvious problems with a certain directory in the repository. Ignoring/Skipping that directory and the check-out as well as the update operation work fine. Looking into the .svn/tmp-directory I do see a single existing file: trz2A87.tmp but obviously there's no trace of the mentioned svn-E78ED003 file. Can you possibly come up with a reproduction recipe that doesn't require access to your repo (unless of course your repo is publicly accessible)? I hope to be able to try that out later today. But no promises. The errors you're getting are coming from the run_file_install() step of the workqueue processor. It's rather hard to understand what's going just from the error messages. Based on what you're saying it seems like the temp file we're trying to move into place doesn't exist. Since looking at the code we try to install the file, and then trap a ENOENT error from the rename call. When the rename call fails with ENOENT we try to create the destination directory. Which fails and thus you see the errors. Right above the code generating the error it creates and writes the temp file. So I would expect to be seeing an error message from that if the file isn't being created. I debugged the code directly and didn't run into the breakpoint I set in run_file_install(). The code-path which returns the error seems to be somewhere else in my case (or did I get u wrong here). From what I can tell, it goes like this: [...] - svn_wc__db_pristine_install() - calls: svn_io_file_rename() - calls: svn_io_file_rename() - calls: apr_file_rename(from_path_apr, to_path_apr, pool); - calls: if (MoveFileExW(wfrompath, wtopath, MOVEFILE_REPLACE_EXISTING || MOVEFILE_COPY_ALLOWED)) That one returns a status code of 720,002. If I'm reading the APR-macros correctly, that corresponds to a window system error of 2 - which according to MSDN means: ERROR_FILE_NOT_FOUND. Setting a breakpoint right before that Move-call in apr_file_rename suggests it's trying to move the following file: G:\Egosoft\X4\Source\.svn\tmp\svn-C5D6631B to G:\Egosoft\X4\Source\.svn\pristine\00\0077903324a83da0419971daeb632ff51529d320.svn-base Looking at the .svn/tmp-directory I do see that a file with the expected file contents was created, but is named differently: trz5B5.tmp From that point, it's obvious that the move-call would fail, since the given filename seems to differ from that on the disk. Taking ur statement into account I assume that SVN tries to actually put a differently named file into that folder and somehow that one seems to get mangled on my system to a different filename. If so, I could try to debug the code a bit further. Would u have the function for me I'd investigate to trace down where the temp-file would be created? One question I do have is have you done anything unusual with how your file system is setup? The .svn/tmp directory needs to be on the same file system as the rest of the working copy since we're doing atomic renames to put files into place. There's nothing special from the file system set-up point of view. The drive is a simple local partition on an IDE drive. There are no symlinks or somesuch in place. File system is NTFS. The issue is reproducible on two different machines, both running Avast Antivirus. I don't know much about the set-up on the other machine, but I would expect it's somehow the same. we recently started getting this error when trying to update or check-out a repository. Weird thing is that the issue only occurs when we try to check-out/update the thing externally (meaning via VPN). So same machine, same working copy doesn't work over the VPN but does without a VPN? Is the working copy perhaps on a network filesystem? No, sry, wasn't clear enough. We didn't test whether it's at all related to a VPN-connection. We can't atm test it without a VPN connection to test whether it's at all related to that. I'll try to do a local set-up of a working copy to see whether it's reproducible in that environment too. Can't tell atm when I got the time to get it done, but might have a few minutes left today. Let's see. Regards, Stefan
Re: How to prevent casual browsing
On 2013-12-01 15:39, Peter Flynn wrote: I have a number of svn repositories running under Apache+subversion on CentOS6/64, with Submin to provide a web GUI to manage them: server.name/svn/foo server.name/svn/bar server.name/svn/blort etc All of them are private; all but one of them are single-user (me) so that I can carry on working from any of my machines in multiple locations. One of them is shared with colleagues on a project: they all have read/write privs on that repo. The URIs are not published or linked, and my colleagues are all well aware of the need to keep their shared URI private. But the requirement is that none of them must be open to casual read access via a web browser, in case someone happen to stumble upon or guess the URI. I am having problems getting the access privs right, as they keep causing svn: E22: Not authorized to open root of edit operation during an svn up. However, in a long exchange with the very helpful submin support (https://ssl.supermind.nl/collab/projects/submin/ticket/336) we have failed to identify settings that work. Currently the svn/conf/authz file says [groups] dev = a,b,c,d,e,me [foo:/] @dev = rw [bar:/] me = rw [blort:/] me = rw The Apache conf.d/subversion.conf says: Location /svn DAV svn SVNParentPath /var/lib/submin/svn # removed GET from LimitExcept to prevent casual browsing LimitExcept PROPFIND OPTIONS REPORT AuthType Basic AuthName Authorization Realm AuthUserFile /etc/svn.auth Require valid-user /LimitExcept /Location and svn.auth specifies a username:encryptedpassword pair for each member of [groups] in the usual way. 1. Browsing with a web browser causes a prompt for the username/password as expected. 2. An svn ci operation works fine. 3. An svn up operation fails, and always causes an E22 error. 4. Replacing the GET in the LimitExcept config allows svn up to work without error, but allows casual browsing of the web interface. Is there a way to prevent the casual browsing while avoiding the E22 error? You do not have AuthzSVNAccessFile $path/to/authz in your Location config. -- olli
Re: server config
On 2013-08-20 01:41, Nico Kadel-Garcia wrote: I think he meant subversion-1.6.11, which is the default version for CentOS 6.4. Check the SELinux settings in /etc/sysconfig/selinux. Set the line to 'SELINUX=permissive' (or disabled) After changing the SELINUX value a reboot is required Additional add a trailing '/' so you config looks so. RewriteEngine on # the trailing '/' in /svn/ is needed to browse repos with standart browser! RedirectMatch ^(/svn)$ $1/ Location /svn/ DAV svn SVNParentPath /var/svn/ # Authentication: Digest AuthName Subversion repository AuthType Digest AuthUserFile /etc/svn-auth.htdigest # Authorization: Authenticated users only Require valid-user /Location On Mon, Aug 19, 2013 at 6:19 PM, Ben Reser b...@reser.org wrote: On 8/19/13 9:07 AM, Scott Frankel wrote: I'm new to SVN server configuration and find myself setting up a CentOS 6.4 server with svn version 1.6.1, following the red-bean book. I'd strongly urge you not to use 1.6.1, see the list of applicable security issues here: http://subversion.apache.org/security/ If you're using the CentOS packages they may have patched those issues without updating the svn version number. You should check that though. If you're setting a new server I wouldn't start with 1.6.x but would go straight to 1.7.x or 1.8.x, probably 1.8.x if you can. I'm having difficulty with authorization /or authentication: my repo appears to be accessible by anyone in spite of requiring valid-user and specifying digest authentication. I believe this because 1) I can download a full working copy of the repo to a 3rd-party logged into a foreign computer, and 2) I have dozens of entries in apache's logfiles, like these from this morning, *prior* to any known/legitimate access to my repos today: svn_logfile: [19/Aug/2013:00:46:32 +] - checkout-or-export / r1 depth=infinity That does indeed look like access without a user. access_log 93.174.93.213 - - [19/Aug/2013:07:23:50 +] GET /w00tw00t.at.blackhats.romanian.anti-sec:) HTTP/1.1 404 319 - ZmEu error_log [Mon Aug 19 07:23:51 2013] [error] [client 93.174.93.213] File does not exist: /var/www/html/MyAdmin These however do not appear to be alarming at all. Neither of them are under the /svn Location on your server where you have put the Require valid-user requirement. They appear to me to be just normal probes run by someone looking for security holes. This sort of thing is just going to be a normal part of running a server on the Internet. Location /svn DAV svn SVNParentPath /var/svn # Authentication: Digest AuthName Subversion repository AuthType Digest AuthUserFile /etc/svn-auth.htdigest # Authorization: Authenticated users only Require valid-user /Location I'm not seeing anything wrong with this, so I'm not sure why you're having a problem. You didn't mention it but I'm wondering what version of httpd you're running, I'm assuming 2.2.x since you're using 1.6.1 on CentOS 6.4.
Re: svnsync 1.8.0 fails if there is a non-printable characters in the commit message
On 2013-06-25 16:02, Daniel Shahaf wrote: Daniel Shahaf wrote on Tue, Jun 25, 2013 at 16:55:24 +0300: Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +: On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote: 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as designed. ;-) No more seriously though, it sure looks like a bug. 0x18 is a perfectly legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII character is encoded as itself in UTF-8 and the first 128 Unicode code points are exactly US-ASCII. Works for me: % svnadmin create r % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' 0 ) r/db/revprops/0/0 mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y That last command should have been (equivalently, but without munging internals): % ln -s =true r/hooks/pre-revprop-change % svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r property 'x' set on repository revision 0 It still works when I use svn:log rather than x as the property name. Daniel Thanks for your first diagnostic, I don't have all the details, the issue is part of an FreeBSD PR where this happened to a user. Since I have the same repository in sync (but with subversion 1.7.10) I've taken a look into the file which breaks the sync for the user with subversion 1.8. It is the following FreeBSD PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=179760 Additional I've ask the OP to join the users@ list so you can get more details from first hand. -- Thanks, olli
Re: svnsync 1.8.0 fails if there is a non-printable characters in the commit message
On 2013-06-26 01:05, Daniel Shahaf wrote: olli hauer wrote on Tue, Jun 25, 2013 at 17:06:35 +0200: On 2013-06-25 16:02, Daniel Shahaf wrote: Daniel Shahaf wrote on Tue, Jun 25, 2013 at 16:55:24 +0300: Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +: On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote: 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as designed. ;-) No more seriously though, it sure looks like a bug. 0x18 is a perfectly legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII character is encoded as itself in UTF-8 and the first 128 Unicode code points are exactly US-ASCII. Works for me: % svnadmin create r % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' 0 ) r/db/revprops/0/0 mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y That last command should have been (equivalently, but without munging internals): % ln -s =true r/hooks/pre-revprop-change % svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r property 'x' set on repository revision 0 It still works when I use svn:log rather than x as the property name. Daniel Thanks for your first diagnostic, I don't have all the details, the issue is part of an FreeBSD PR where this happened to a user. Since I have the same repository in sync (but with subversion 1.7.10) I've taken a look into the file which breaks the sync for the user with subversion 1.8. It is the following FreeBSD PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=179760 So it indeed is FreeBSD - I thought the revnums looked familiar. I'll note the path of least resistance to getting svnsync to work is to ask someone with access to edit the log message on svn.freebsd.org. Daniel Additional I've ask the OP to join the users@ list so you can get more details from first hand. Hi Daniel, it turns out the OP was not using subversion from ports and the original apache subversion 1.7.x/1.8 has not this issue. Sorry for the noise and thanks for your help! -- Regards, olli
svnsync 1.8.0 fails if there is a non-printable characters in the commit message
svnsync 1.8.0 fails if there is a non-printable characters in the commit message Error message: svnsync: E22: Safe data 'MFC r251249, r251251, r251252, r' was followed by non-ASCII byte 24: unable to convert to/from UTF-8 No issue with svnsync 1.7.10 (neon based) Parts of the file (synced with svnsync 1.7.10) $ cat $repo/db/revprops/251/251614 ... MFC r251249, r251251, r251252, r2512, r251254 and r251515: $ more $repo/db/revprops/251/251614 ... MFC r251249, r251251, r251252, r^X2512, r251254 and r251515: $ hexdump -C $repo/db/revprops/251/251614 ... 0070 31 2c 20 72 32 35 31 32 35 32 2c 20 72 18 32 35 |1, r251252, r.25| Culprit is the hex(18) sign. -- olli
Re: large db/revs files
On 2013-06-04 16:30, Andre Harper wrote: Hi, all – I am not subscribed and would appreciate being explicitly Cc:ed in any responses. I am on a team that began using subversion near the end of last year. As a part of our process, we tag each successful run of our systems. This can mean thousands of tags for certain systems every six months. We’re having an issue with the db/revs directory size, which for all our projects currently exceeds 289G. We only use relatively small working directories containing less than a meg of text files; no binary files. In the archives, I found a mention that the db/revs directories are populated using xdelta, but there didn’t appear to be a solution to large file sizes at time [http://svn.haxx.se/users/archive-2011-08/0229.shtml ]. I was hoping someone may have found a work-around or solution. Would someone be able to: 1)suggest how to avoid this in the future 2)suggest how to reduce the current large files (if possible) Thank you. André Harper Wow, how many revisions are this and what is the average size of the source (1MB)? Even with a several thousands tags and plain text files there is a change to keep the repository small. For example your files could be plain text files with - max 10 chars per line (good for xdelta) - containing only single line (bad for xdelta) - EOL style / white spacing changed during commit (bad for xdelta) - With the worst case examples in mind you can inspect your sources and maybe find some improvements. Have you looked direct on the server side to the repo ($repo/db/), where is all the space used? - rep-cache.db - revs - transactions (I've seen repos with several GB inside) - revprops
Re: Parmently removing directory from server to make space
On 2013-03-26 12:25, Daniel Shahaf wrote: Anil Bakshi wrote on Tue, Mar 26, 2013 at 16:37:25 +0530: So I will go with your suggestion: svndumpfilter exclude /E_Learning/Development/Project1 repo.dump filteredDump.dump Please correct me. You might try this command: % grep -a '^Node-path:' repo.dump | head to see what paths inside the dump file look like. .oO ( maybe we should have an 'svndumpfilter info' command? I suppose it could basically cat the dumpfile, except: file reps would be omitted; dir reps would be omitted (unless we figure out something sensible to dowith them, eg 5 children); properties would be omitted (but in the future we could list propnames or propnames and propvalues). ) Here the script normalize-dump.py (in the svn tar file) comes is really handy. From my own experience with big dumps it is ways faster to process the dump with this script and then grep for Node-path/Node-copyfrom-path. Additional hint for Windows users: run svndumpfilter this way: python -u $path/svndumpfilter.py
Re: Need to restructure repo folders: Problem: SVN COPY is recursive
On 2013-03-15 21:07, tim.willi...@ucb.com wrote: I am not very good at giving examples on a Friday afternoon, admittedly. I will try again. Original Folders \Barn\livestockNames.txt \Barn\chickens\chickenNames.txt \Barn\chickens\food\chickenFeed.txt New Structure I want: \NewBarn\livestockNames.txt \NewBarn\birds\chickenNames.txt \NewBarn\birds\chickenFeed.txt If I use SVN COPY to copy chickenNames.txt to the new folder: svn copy \Barn\chickens\ \NewBarn\birds\ I will get: \NewBarn\birds\food\chickenFeed.txt and I don’t want that folder called \food and its content. Hm, why do you copy the folder instead the single file ? svn copy \Barn\chickens\chickenFeed.txt \NewBarn\birds\ If you have a bunch of files you can do this quickly with a wrapper script
Re: Need to restructure repo folders: Problem: SVN COPY is recursive
On 2013-03-15 21:30, tim.willi...@ucb.com wrote: Yes, it appears I am headed toward a wrapper script to copy one file at a time. I wanted to make sure I was not missing something in SVN that would make it easier (a non-recursive copy, or something in svnmucc where I could copy a bunch a files and commit them all in a single new revision, for example). It appears that I am stuck creating a new revision for every single file I need to move. This will make the SVN Log history long and boring, but it appears there is not much else that can be done if my users want to retain the development history. Otherwise I would just SVN export it, remap it in a work area and commit it all in one big go. Thanks for the sanity check, folks. Tim Maybe something to test (but maybe totally wrong ...) svn move \Barn\chickens\food \tmp\ svn copy \Barn\chickens\ \NewBarn\birds\ svn revert \Barn\chickens\food svn commit Increasing the revision will be done only if you do a commit.
Re: Working Copy on Network / Usage of multiple users
On 2012-12-27 16:03, Karl Heinz Marbaise wrote: Hi, i just want to check my assumption by the user/dev community. I have the following scenario. From a central SVN server there should be done checkouts (different working copies) on a network drive. The working copies will be accessed from different users which means a thing like this: Network: working-copy-1 User 1 accesses working-copy-1 and changes some files and will checkin those changes. User 2 accesses working-copy-1 and changes some files and will checkin those changes. etc. My knowledge is that if those accesses to the above working copy can be handled strictly sequential it might work but i'm not 100% sure..(may be someone can give more detailed informations).. Apart from that I expect failures if a working copy will be accessed in parallel from different users at the same time on the network based on the pristine-copy information and other meta-data which is stored in the .svn folder (SQLite database etc.). Can someone give more detailed information if the above assumption is right or wrong ? I assume it's right... Many thanks in advance. Kind regards Karl-Heinz Marbaise I'm not sure if I got your scenarios right ... - the working-copy-1 is located on a network share which is mounted by other users in read/write mode - user 1 - user N has mounted the working copy and modifies files there, so this changes are direct transparent to all others without svn update - user 2 -- same as user 1 -- My question for this scenario is then more in the direction from which location in the working copy will user1/2 commit/update? Or is someone the svn master who updates/commits the changes? About the wc.db, even on a network share the database should be locked exclusive for write, so no other can do update/commits at the same time. I expect for this scenario more failures in the way how a single file will be locked during the work (two users are working on the same file, in worst case with notepad that does not lock the file and does not recognize changes) -- Regards, olli
Re: Is there a way to dump the checksums from a svn repo?
On 2012-11-29 20:13, Philip Martin wrote: Daniel Shahaf d...@daniel.shahaf.name writes: Philip Martin wrote on Thu, Nov 29, 2012 at 18:26:04 +: Daniel Shahaf d...@daniel.shahaf.name writes: Les Mikesell wrote on Thu, Nov 29, 2012 at 09:59:47 -0600: But, the copy built by svnsync doesn't necessarily get stored the same way, does it? I think in 1.8/fsfs it will byte-for-byte identical. (except rep-cache.db, but you can remove that file without consequences) There was a dev@ thread by philipm about this not too long ago. No, an svnsync mirror is usually not identical to the master. It does contain the same versioned data but the representation of that data is different. For example, every failed commit on the master will bump the fsfs sequence number and that will cause the node-revision-ids to be different. Node-revision-id's in revisions don't embed transaction id's... For example the noderev header (yes, header, not just id) of /subversion/trunk/notes is identical between svn.us and svn.eu. OK. But the sequence number differences do show up in other places: svnadmin create repo svn mkdir -mm file://`pwd`/repo/A # r1 svn mkdir -mm file://`pwd`/repo/A # fail svn mkdir -mm file://`pwd`/repo/A/B # r2 svnadmin create repo2 svnadmin dump repo | svnadmin load repo2 diff repo/db/revs/0/2 repo2/db/revs/0/2 37c37 _1.0.t1-2 add-dir false false /A/B --- _1.0.t1-1 add-dir false false /A/B Further, node-revision-ids can vary for other reasons. Representations in the revision files are in whatever order the client sends representations to the server. There are no defined orders for clients to use so it is quite likely that commits to the master and the mirror will use different orders: mkdir zz echo foo zz/f echo bar zz/g echo zigzig zz/F echo zagzag zz/G svnadmin create repo svn mkdir -mm file://`pwd`/repo/A svnadmin create repo2 svnsync init file://`pwd`/repo2 file://`pwd`/repo svnsync sync file://`pwd`/repo2 I see orders: repo/db/revs/0/1: foo, zigzig, zagzag, bar repo2/db/revs/0/1: zigzig, zagzag, foo, bar That affects the offsets in the text: lines, often changing the line length, which in turn affects the position of the subsequent nodes, and the position of the nodes affects the node-revision-ids. Thats what I also see with svnsync, specially for revisions with a lot of files in the initial commit (master and mirror are the same OS and installed with exact the same packages no matter if I sync over svn or http(s)).
Re: Is there a way to dump the checksums from a svn repo?
On 2012-11-29 19:24, Philip Martin wrote: olli hauer oha...@gmx.de writes: Is there a way to dump the checksums from a svn repo? What I'm doing at the moment on masters and slaves is $ svnadmin verify and $ sqlite $repo/db/rep-cache.db select hash,revision from rep_cache then additional comparing the sqlite output from master and slaves. Since rep-cache is not used during read requests it would be nice to have for example a parameter for svnadmin verify to output the checksums so they can be compared between master and slaves. Is there way for example via the python/perl API? Thanks for every answer and code snippet ... I did it in C but I suppose you might be able to use the Python bindings. I did svn_fs_open() svn_fs_revision_root(N) svn_repos_replay2(N-1) which drove an editor from rN-1 rto rN and the editor did nothing except extract the checksum from the close_file callback. Thanks for the hint, I will do some tests with your promised snipped.
Re: Is there a way to dump the checksums from a svn repo?
On 2012-11-25 21:49, Thorsten Schöning wrote: Guten Tag olli hauer, am Sonntag, 25. November 2012 um 20:18 schrieben Sie: Thanks for every answer and code snippet ... I'm interested in which problem you try to solve with your approach? What's the reason behind it? Maybe there are other ways to accomplish what you want. Mit freundlichen Grüßen, Thorsten Schöning Sorry for the delay ... I will try to explain some of my thoughts. Given you have one svn master from where dedicated slaves are syncing Both master and first slaves are under your control so far so good. Now some additional mirrors which are not under you full control are syncing from the slaves to help offload traffic. Someone hacks one of the additional mirrors, modifies a revision and adjust the checksum (as described on many places how-to fix a corrupt repo) so it looks OK even with svnadmin verify. Now if you have a million of revisions it will be hard to detect such an issue. Wouldn't it be nice to have the ability to calculate the checksums regularly so they can be compared with the upstream checksums? Another methode to detect such thing would be rsync the repo first with a dry-run and then do a live sync but svnsync is preferred. -- Regards, olli
Is there a way to dump the checksums from a svn repo?
Is there a way to dump the checksums from a svn repo? What I'm doing at the moment on masters and slaves is $ svnadmin verify and $ sqlite $repo/db/rep-cache.db select hash,revision from rep_cache then additional comparing the sqlite output from master and slaves. Since rep-cache is not used during read requests it would be nice to have for example a parameter for svnadmin verify to output the checksums so they can be compared between master and slaves. Is there way for example via the python/perl API? Thanks for every answer and code snippet ... -- Regards, olli
