Re: Post commit script examples?
On Friday, August 20, 2010 09:42:44 am Ryan Schmidt wrote: > > Also imagine the logs: Bob did a fix, hook script did a fix. Robert > > did a fix, hook script did a fix. Alice did a fix, hook script did a > > fix. Every other log entry is your hook script modifying code. > > Only commits immediately after adding a new file will be a commit by the > hook script, to add the property. For commits that just modify existing > files, there will be no need for the hook script to commit anything. It could be done within the same commit, actually, without a separate commit. Here is a simple pre-commit in Python that will set 'foo:bar' property to 'baz' on all committed files but not directories. [[[ #! /usr/bin/python # vi: set sw=2 : import sys from svn import core, fs, delta, repos def v(pool, repos_path, txn): fs_ptr = repos.fs(repos.open(repos_path, pool)) txn_ptr = fs.open_txn(fs_ptr, txn, pool) txn_root = fs.txn_root(txn_ptr, pool) chg = fs.paths_changed2(txn_root, pool) for f in chg: if chg[f].node_kind == core.svn_node_file: fs.change_node_prop(txn_root, f, "foo:bar", "baz") if __name__ == '__main__': core.run_app(v, sys.argv[1], sys.argv[2]) ]]] Here is how it works. Note the foo:bar property change. [[[ /tmp$ svnadmin create aa /tmp$ cp pre-commit aa/hooks/ /tmp$ svn co file:///tmp/aa awc Checked out revision 0. /tmp$ cd awc /tmp/awc$ echo qwerty > qwerty /tmp/awc$ svn add qwerty A qwerty /tmp/awc$ svn ci -m 1 qwerty Adding qwerty Transmitting file data . Committed revision 1. /tmp/awc$ svn di -c 1 Index: qwerty === --- qwerty (revision 0) +++ qwerty (revision 1) @@ -0,0 +1 @@ +qwerty Property changes on: qwerty ___ Added: foo:bar ## -0,0 +1 ## +baz ]]] The problem with this approach is that the working copy is not aware that the commit was somehow altered on the server, so the client does not see this property on the file: [[[ /tmp/awc$ svn pl qwerty /tmp$ ]]] Therefore, I would only use this approach for properties that client does not care about, and svn:needs-lock is not one of such properties. Regards, Alexey.
Non-obvious behavior of 'svn merge' when using old repository
Hi all, I have just stumbled over a not user-friendly behavior of 'svn merge'. The environment was: - server running 1.6.11 - repository has been created under 1.4.2 and has not been run through 'svnadmin upgrade' since - client 1.6.6 When attempting to sync up a branch with latest trunk changes, 'svn merge' attempted to bring over ALL revisions of trunk, even those before the branchpoint. This, obviously, failed with a lot of tree conflicts. $ svn merge ^/trunk --- Merging r2 through r578 into '.': I finally figured out that this is due to outdated repository version. However, shouldn't svn provide some warning in this case to indicate that it were not able to determine which revisions need to be merged? Note that 'svn merge --reintegrate' detects this condition and emit the warning: $ svn merge --reintegrate ^/branches/ svn: Retrieval of mergeinfo unsupported by '' Shouldn't regular 'svn merge' do the same? I am not sure if it was fixed in more recent versions of Subversion. Regards, Alexey.
Re: First Hands-on Subversion—Where/How?
Dear Pietro, I think you should start your learning Subversion from learning what a "command line" is. Subversion manual is no substitute for knowing the OS. Or alternatively, have someone install a repostory for you and go with a GUI client, such as TortoiseSVN. Regards, Alexey. On Friday, October 28, 2011 03:52:31 pm Pietro Moras wrote: > more specific questions > > > > My pleasure, dear > Geoff, Here you have some > very Specific Questions. > > SQ1] How to get what I > presume is a nice Subversion prompt: > > $ >on one of my > standard Windows machines, so to test the wonderful Subversion > commands so eloquently described by the mentioned self-declared > Official Guide and Reference Manual, so practically useless at the > very beginning of a learning process; that is, exactly when you need > most practical and effective information and support? > > > SQ2] Why should I go > scrabbling and begging via Google for practical, operative info, I'd > reasonably expected to find right away at page 1 on the mentioned > book, or at the page 1 on the Subversion web site? > > > SQ3] Am I the first > Subversion potential user starting from scratch? >Everybody else > knowing how to set-up a Subversion environment even before beginning > to use it? > > Of course thank you for > pointing me to the right direction. Of course. > > All the best. Yours, - P.M. > > Date: Fri, 28 Oct 2011 11:20:54 -0700 > Subject: Re: First Hands-on Subversion—Where/How? > From: ghoff...@cardinalpath.com > To: studio...@hotmail.com > CC: users@subversion.apache.org > > > > On Fri, Oct 28, 2011 at 11:00 AM, Pietro Moras > wrote: > > > > > > > > > > > > Dear Subversion > cognoscenti, > > Seriously > intentioned to explore what Subversion is all about, armed with good > will and a good reference book (“Version Control with Subversion”, > by Ben Collins-Sussman, Brian W. Fitzpatrick, C. > Michael Pilato), > I got immediately lost & stuck at the very first command: > > $ > svn help > > > > ?] That said, where/how on earth > could I get such Subversion grass-roots commands working?Is there any > practical way, any practical tool, any practical good soul/good > organisation where to find a test client-server setup where to > (seriously) “play” with Subversion VCS? I'd be happy to begin > even with a tiny a client-server set-up onto a machine of mine, would > such a tool available; even no idea whether such a naïve idea of > mine is feasible or not. > > Gratefully yours, > > - > P.M. > > > Ref.: > dr. Pietro Moras > Email > studio...@hotmail.com > > > > Greetings Pietro, > Start here http://svnbook.red-bean.com/ > Then I would recommend a Google search like install subversion > {platform_or_distro_you_use} -- for example here's a good quick overview > at Ubuntu https://help.ubuntu.com/community/Subversion > > Remember, `Subversion` is both a client software and a server software, so > it doesn't do you much good to learn `svn ___` (client) unless you have > already used svnadmin to create a repository... (server). You can do this > all locally, or on a VM (I would recommend go this route, eg virtualbox, > if you're on Windows). > > Post back here with more specific questions as you go.
Making merge "replay" branch changes on reintegrate?
Hi all, I wonder if there is a way to make "svn merge --reintegrate" replay actual changes that happened on the branch instead of just copying the sources from the branch. What I mean is: 1. I have some project in /trunk 2. To do some experimental feature, I copy it to an engineering branch: $ svn cp $URL/trunk $URL/engr/mybranch 3. On that branch, I do some renames followed by some changes, e.g.: $ svn mv include/foo.h include/bar.h $ vi include/bar.h 4. I commit the changes on the branch, merge with trunk. 5. Now when I try to reintegrate the changes to the trunk with "svn merge -- reintegrate", I see that include/bar.h is indeed added with history ("A +"), but it was copied from ^/engr/mybranch/include/bar.h, not from ^/trunk/include/foo.h (as it was on the branch, at step #3 above). The reason for this request is that in our project, all changes on the trunk must be peer-reviewed, but engineering branches are considered "private sandboxes" where one may do whatever changes he likes. In the scenario above, bar.h would have a history going through the engineering branch, which in certain cases would make peer reviews inconvenient (as the engineering branch may include some spurious changes that did not make into the final merge). So, is it possible to either - make "svn merge --reintegrate" apply the same changes to the trunk rather than copy the sources from the branch? OR - somehow change the "copyfrom" information on a file before checking it in (so that the result of merging could be adjusted before being checked in)? Regards, Alexey.
svn: Attempted to get textual contents of a *non*-file node
Hi all, I ran into the following error message with Subversion: svn: Attempted to get textual contents of a *non*-file node The issue, as pointed out by email thread [1], is that the directory being merged contains a file with the same name as the directory. I.e., there is /trunk/foo directory containing /trunk/foo/foo file. However, even if I tried the suggestion from Ben Collins-Sussman, it didn't help: 'svn merge' still complained with the same error message. Reproduction script attached. Is there a way such projects can use 'svn merge' command? I tried with Subversion trunk and, although the error message is different ("svn: E160017: '/trunk/foo' is not a file"), the result is still the same. While it is a better message than the one in 1.6, it still does not explain why Subversion expects /trunk/foo to be a file for the following commands: svn merge -c 4 ^/trunk/foo svn merge -c 4 ^/trunk/foo . svn merge ^/trunk/foo@3 ^/trunk/foo@4 . svn merge -r 3:4 ^/trunk/foo . As another side note, Subversion leaves behind a zero-sized temporary file created for the merge. Regards, Alexey. [1] http://markmail.org/message/qqh3r6d4tcdyjnz2#query: +page:1+mid:vcjektlfn37mxyld+state:results x.sh Description: application/shellscript
Re: svn: Attempted to get textual contents of a *non*-file node
Hi Daniel, On Monday, March 05, 2012 11:33:33 pm Daniel Shahaf wrote: > Alexey Neyman wrote on Mon, Mar 05, 2012 at 16:14:24 -0800: > > Hi all, > > > > I ran into the following error message with Subversion: > > > > svn: Attempted to get textual contents of a *non*-file node > > > > The issue, as pointed out by email thread [1], is that the directory > > being merged contains a file with the same name as the directory. I.e., > > there is /trunk/foo directory containing /trunk/foo/foo file. > > > > However, even if I tried the suggestion from Ben Collins-Sussman, it > > didn't help: 'svn merge' still complained with the same error message. > > Reproduction script attached. Is there a way such projects can use 'svn > > merge' command? > > > > I tried with Subversion trunk and, although the error message is > > different ("svn: E160017: '/trunk/foo' is not a file"), the result is > > still the same. While it is a better message than the one in 1.6, it > > still does not explain why Subversion expects /trunk/foo to be a file > > for the following commands: > > > > svn merge -c 4 ^/trunk/foo > > svn merge -c 4 ^/trunk/foo . > > svn merge ^/trunk/foo@3 ^/trunk/foo@4 . > > svn merge -r 3:4 ^/trunk/foo . > > Yeah, I just tried with trunk, couldn't get the merge to work, with > those commands (some of which are made equivalent by the argument > parser) or with . > > Looks like a bug to me, assuming it works when the dir and the file are > not both named the same thing. I confirm it works when dir and file do not have the same name. E.g., if you rename new file 'B' in the attached testcase to, say, 'X' - it passes. There is one more condition for this bug to manifest: the offending directory must be the current directory: even though $ svn merge -c 4 ^/trunk/foo . fails, the following $ wc=`pwd` $ cd .. $ svn merge -c 4 ^/trunk/foo $wc works. I guess, it's sort of a workaround. > > As another side note, Subversion leaves behind a zero-sized temporary > > file created for the merge. > > And this one too. (the file is in the wc root) > > > Regards, > > Alexey. > > > > [1] http://markmail.org/message/qqh3r6d4tcdyjnz2#query: > > +page:1+mid:vcjektlfn37mxyld+state:results > > Could you file an issue? Perhaps send a patch adding a regression test > for this (in Python)? (See subversion/tests/cmdline/README) Issue 4139 created. Attached is a patch that adds an XFail to the test suite for this issue. Regards, Alexey. Index: subversion/tests/cmdline/merge_tests.py === --- subversion/tests/cmdline/merge_tests.py (revision 1297725) +++ subversion/tests/cmdline/merge_tests.py (working copy) @@ -17467,6 +17467,64 @@ 'merge', sbox.repo_url + '/A', A_COPY_path) +@XFail() +@Issue(4139) +def merge_dir_file_same_name(sbox): + "merged directory has file with same name" + sbox.build() + wc_dir = sbox.wc_dir + + # Paths of interest in first WC + A_B_B_path = os.path.join(wc_dir, 'A', 'B', 'B') + + # Create file 'B' in path '/A/B' - revision 2 + svntest.main.file_write(A_B_B_path, "Project B now has file B") + svntest.main.run_svn(None, 'add', A_B_B_path) + expected_output = svntest.wc.State(wc_dir, { +'A/B/B' : Item(verb='Adding'), +}) + expected_status = svntest.actions.get_virginal_state(wc_dir, 1) + expected_status.add({ +'A/B/B' : Item(status=' ', wc_rev=2), +}) + svntest.actions.run_and_verify_commit(wc_dir, expected_output, +expected_status, None, +wc_dir) + + # Copy '/A' to '/A_COPY' - revision 3 + sbox.simple_repo_copy('A', 'A_COPY') + + # Now modify '/A/B/B' - revision 4 + svntest.main.file_write(A_B_B_path, "File B is now modified") + expected_output = svntest.wc.State(wc_dir, { +'A/B/B' : Item(verb='Sending'), +}) + expected_status = svntest.actions.get_virginal_state(wc_dir, 1) + expected_status.add({ +'A/B/B' : Item(status=' ', wc_rev=4), +}) + svntest.actions.run_and_verify_commit(wc_dir, expected_output, +expected_status, None, +wc_dir) + + # Now check out A_COPY/B in a separate WC + wc_copy = sbox.add_wc_path('copy') + svntest.actions.run_and_verify_s
Re: SVN Problems
Hi Waseem, On Tuesday, March 06, 2012 06:57:19 am Waseem Shahzad wrote: > Hi Guys > >I am running SVN in the .Net environment. à Using TSVN > and Ankh SVN > > > > I have checkout the repository (almost 1 GB) and working in it. It gets > stuck when on Commit with followings solutions: > > > > 1. It shows locked status even other people not being used it in > Lock. > > 2. I will have to run CleanUp command , Why? > > 3. After moving from VSS to SVN , Visual Studio gets slower down in > saving files , Why? > > > > Cheers, > > Waseem Bukhari > > CMer With regard to #1 and #2, I think you're mixing two separate "locking" concepts. One is "locking" in the repository - which prevents other developers from working on a given file that's being modified by another person. Another is "locking" the working copy during certain operations (such as commit) to prevent concurrent changes in the working copy (i.e. adding new files). As to why it gets "stuck in commit" - is it possible that you have some access control scripts on the repository side? Running 1GB commit through them may take substantial time, depending on what those scripts do. Such repository- side scripts may also be the reason why saving files is slower. Regards, Alexey.
Directory as part of the changelist - possible?
Hi all, Is it possible to add a directory to a change list? I have a property modification on one of the directories - however, an attempt to include it in the changelist fails: $ svnadmin create /tmp/foo $ svn co file:///tmp/foo foo.wc Checked out revision 0. $ cd foo.wc $ svn mkdir dir1 A dir1 $ svn ci -m 1 Adding dir1 Committed revision 1. $ svn ps foo:bar baz dir1 property 'foo:bar' set on 'dir1' $ svn st M dir1 $ svn cl cl0 dir1 Skipped 'dir1' $ svn --version svn, version 1.8.0-dev (under development) compiled Nov 16 2011, 13:23:31 Regards, Alexey.
Re: FreeBSD project and subversion.
On Thursday, January 31, 2013 09:08:20 am Stefan Sperling wrote: > BTW, I went over one of FreeBSD's svn wiki pages a while back, and added > answers to open questions on this page: > https://wiki.freebsd.org/SubversionMissing Speaking of which: Is there a reason why it is necessary to use "the svn diff --old ^/somebranch --new ." instead of the intuitive "svn ^/somebranch ."? It has bugged me for a while until I discovered this --old/--new trick. Regards, Alexey.
Re: FreeBSD project and subversion.
On Monday, February 04, 2013 10:17:29 PM Stefan Sperling wrote: > On Mon, Feb 04, 2013 at 11:54:21AM -0800, Alexey Neyman wrote: > > On Thursday, January 31, 2013 09:08:20 am Stefan Sperling wrote: > > > BTW, I went over one of FreeBSD's svn wiki pages a while back, and added > > > answers to open questions on this page: > > > https://wiki.freebsd.org/SubversionMissing > > > > Speaking of which: > > > > Is there a reason why it is necessary to use "the svn diff --old > > ^/somebranch --new ." instead of the intuitive "svn ^/somebranch ."? > > Would "svn diff ^/somebranch ." be synopsis 1? > > svn diff -rN:M ^/somebranch@HEAD .@HEAD It cannot be "synopsis 1", as its description states that "TARGETs may be all working copy paths or all URLs". In this case, we have one of each. Moreover, synopsis 1 says that for URLs, -r or -c must be present - and my example has no -r/-c option. So, the only possible use of this syntax would be synopsis 2: > svn diff --old ^/somebranch@HEAD --new .@HEAD > (i.e. show the difference between the two (path, revision) pairs > somebranch@HEAD and .@HEAD) ? > > > It has bugged me for a while until I discovered this --old/--new trick. > > I agree that --old --new syntax isn't intuitive at all. What adds oil to the fire is that the error message, which claims that "Target lists to diff may not contain both working copy paths and URL". If instead it said that the supplied syntax is ambiguous and suggested to use --new/--old for disambiguation, this would have saved a lot of frustration and would eliminate the need of "svn compare" subcommand :) For example: svn: Ambiguous target list (contains both work copy paths and URLs). Use --old/--new syntax to disambiguate. Regards, Alexey.
Re: FreeBSD project and subversion.
On Tuesday, February 05, 2013 05:08:40 PM Stefan Sperling wrote: > On Tue, Feb 05, 2013 at 12:59:43PM +0100, Johan Corveleyn wrote: > > Given what I said above, this can't be done in general (not if you > > give two URLs or two wc paths, because then both synopsis 1 and 2 are > > possible -- it would obviously break backwards compatibility if we > > would now start erroring out on these, while these used to work fine > > with automatically picking usage 1 before). Only if one of them is a > > URL and the other a WC path, because we give an error message then > > anyway. > > I think Alexey meant only the cases where we currently error out, i.e. where > exactly two targets are given and one is a path and the other is a URL. > > I didn't realise this but there is indeed no ambiguity in allowing this > syntax as a shorthand for a corresponding '--old X --new Y' diff invocation. > > Thanks Alexey, see http://svn.apache.org/r1442640 > Making these invocations produce a diff instead of an error makes things > less surprising for users. Thanks! There is one more weird issue with svn diff, see the script below. The issue is that "--old=A --new=B" is not opposite of "--old=B --new=A". I don't know if it is a bug or another ambuguity I am not aware of :) Here is the script: [[[ #!/bin/bash REPO=/tmp/foo WC=/tmp/foo.wc rm -rf $REPO $WC svnadmin create $REPO svn co -q file://$REPO $WC cd $WC echo r1 > a svn add -q a svn ci -q -m R1 echo r2 > a svn ci -q -m R2 svn up -q -r 1 echo new > a echo "Issue: --old=A --new=B is not opposite of --old=B --new=A" echo " Running: svn di --old=^/ --new=." svn di --old=^/ --new=. echo " Running: svn di --old=. --new=^/" svn di --old=. --new=^/ ]]] And here is the output (svn 1.7.6): [[[ Issue: --old=A --new=B is not opposite of --old=B --new=A Running: svn di --old=^/ --new=. Index: a === --- a (.../file:///tmp/foo) (revision 2) +++ a (working copy) @@ -1 +1 @@ -r2 +new Running: svn di --old=. --new=^/ Index: a === --- a (working copy) +++ a (.../file:///tmp/foo) (revision 2) @@ -1 +1 @@ -r1 +r2 ]]] Regards, Alexey.
Re: FreeBSD project and subversion.
On Tuesday, February 05, 2013 12:59:43 PM Johan Corveleyn wrote: > On Tue, Feb 5, 2013 at 1:39 AM, Alexey Neyman wrote: > > On Monday, February 04, 2013 10:17:29 PM Stefan Sperling wrote: > >> On Mon, Feb 04, 2013 at 11:54:21AM -0800, Alexey Neyman wrote: > >> > Is there a reason why it is necessary to use "the svn diff --old > >> > > >> > ^/somebranch --new ." instead of the intuitive "svn ^/somebranch ."? > >> > >> Would "svn diff ^/somebranch ." be synopsis 1? > >> > >> svn diff -rN:M ^/somebranch@HEAD .@HEAD > > > > It cannot be "synopsis 1", as its description states that "TARGETs may be > > all working copy paths or all URLs". In this case, we have one of each. > > Moreover, synopsis 1 says that for URLs, -r or -c must be present - and my > > example has no -r/-c option. > > Sorry, that's not true. From 'svn help diff' of 1.7: > > diff (di): Display the differences between two revisions or paths. > usage: 1. diff [-c M | -r N[:M]] [TARGET[@REV]...] >2. diff [-r N[:M]] --old=OLD-TGT[@OLDREV] [--new=NEW-TGT[@NEWREV]] \ >[PATH...] >3. diff OLD-URL[@OLDREV] NEW-URL[@NEWREV] > > So for usage 1 the -c and -r are optional. You didn't quote the second part of 'svn help diff' description of synopsis 1. Here it is: 1. Display the changes made to TARGETs as they are seen in REV between two revisions. TARGETs may be all working copy paths or all URLs. If TARGETs are working copy paths, N defaults to BASE and M to the working copy; if URLs, N must be specified and M defaults to HEAD. The '-c M' option is equivalent to '-r N:M' where N = M-1. Using '-c -M' does the reverse: '-r M:N' where N = M-1. If you read it, you'll see that (a) Mix of URLs and paths are not allowed for this synopsis (b) For URLs, either -r or -c must be provided Regards, Alexey. y.
Re: FreeBSD project and subversion.
On Tuesday, February 05, 2013 01:55:20 PM Lathan Bidwell wrote: > On 02/05/2013 01:14 PM, Stefan Sperling wrote: > > On Tue, Feb 05, 2013 at 09:19:19AM -0800, Alexey Neyman wrote: > >> There is one more weird issue with svn diff, see the script below. The > >> issue is that "--old=A --new=B" is not opposite of "--old=B --new=A". I > >> don't know if it is a bug or another ambuguity I am not aware of :) > >> > >> Here is the script: > >> [[[ > >> #!/bin/bash > >> > >> REPO=/tmp/foo > >> WC=/tmp/foo.wc > >> > >> rm -rf $REPO $WC > >> svnadmin create $REPO > >> svn co -q file://$REPO $WC > >> cd $WC > >> > >> echo r1 > a > >> svn add -q a > >> svn ci -q -m R1 > >> echo r2 > a > >> svn ci -q -m R2 > >> svn up -q -r 1 > >> echo new > a > >> echo "Issue: --old=A --new=B is not opposite of --old=B --new=A" > >> echo " Running: svn di --old=^/ --new=." > >> svn di --old=^/ --new=. > >> echo " Running: svn di --old=. --new=^/" > >> svn di --old=. --new=^/ > >> ]]] > >> > >> And here is the output (svn 1.7.6): > >> [[[ > >> Issue: --old=A --new=B is not opposite of --old=B --new=A > >> > >>Running: svn di --old=^/ --new=. > >> > >> Index: a > >> === > >> --- a (.../file:///tmp/foo) (revision 2) > >> +++ a (working copy) > >> @@ -1 +1 @@ > >> -r2 > >> +new > >> > >>Running: svn di --old=. --new=^/ > >> > >> Index: a > >> === > >> --- a (working copy) > >> +++ a (.../file:///tmp/foo) (revision 2) > >> @@ -1 +1 @@ > >> -r1 > >> +r2 > >> ]]] > >> > >> Regards, > >> Alexey. > > > > I can reproduce this with a trunk build. Can you please file an issue > > for this? I'm not going to get to this right away. Thanks! > > Here is the issue that I see: > > The --old=. get's the workspace version of ., but --new get's the > non-changed version of /. > > So, I believe it is comparing "r1" with the r2 contents of "r2" and not > comparing both workspace versions. > > RevContents > 1r1 > 2r2 > 2* new > > 2* only get's referenced when it is in the --old position, > 2 get's used when its in the --new position. > > Please correct me if I'm wrong about this, but that is what seems to be > a logical reasoning behind that behavior. First, it is just the opposite: 2* gets referenced when it is in the --new, not --old position. Second, it is not the reasoning (why it behaves so) but rather a description of current behavior, which I think was rather obvious from the example script. The reason for this behavior is that the code in diff-cmd.c makes the following defaults for the old/new revisions: if (opt_state->start_revision.kind == svn_opt_revision_unspecified) opt_state->start_revision.kind = svn_path_is_url(old_target) ? svn_opt_revision_head : svn_opt_revision_base; if (opt_state->end_revision.kind == svn_opt_revision_unspecified) opt_state->end_revision.kind = svn_path_is_url(new_target) ? svn_opt_revision_head : svn_opt_revision_working; These defaults make some sense, given that new target is optional and defaults to old target if not specified. If new target is specified, and is different from from old target, I am not so sure if it makes sense. Note that there is a special case if both old/new targets are WC paths - in that case, both old and new targets default to svn_opt_revision_working. So, $ svn diff --old=foo --new=bar will compare modified version of 'foo' against modified version of 'bar' [*], but $ svn diff --old=foo --new=^/bar would compare base version of 'foo' against HEAD version of 'bar. Does it make sense? I would say, if both old and new are specified, old target's revision should default to 'working' as well. Problem is further aggravated since there is no commandline alias to request svn_opt_revision_working setting from the command line: only 'head', 'prev', 'base' and 'committed' are currently available. Hence, it is not possible to request the exact opposite of the patch using -rN:M syntax if one of the -- old/--new targets is a working copy path. Is t
Merge error with SVN 1.8.0
Hi, I've tried upgrading the client to SVN 1.8, and now see some strange merge errors while reintegrating the branch. According to http://subversion.apache.org/docs/release-notes/1.8.html#auto-reintegrate the --reintegrate option is now deprecated, its use is discouraged and SVN should be able to figure that out automatically. However, when I tried a plain "svn merge", it gave me the following error: [aneyman@build2 trunk]$ svn merge ^/MERGE-PATH svn: E210002: Unable to connect to a repository at URL 'svn://MERGE-URL' svn: E210002: Network connection closed unexpectedly Strangely, 'svn merge --reintegrate' worked fine. We are running 1.6.11 on the server (stock RedHat RPM, "1.6.11-2.el6_1.4" version). I installed SVN 1.8.0 RPM from WanDisco ("1.8.0-1") on the client. Any clues/suggestions as to how to debug this further? Regards, Alexey.
Re: Merge error with SVN 1.8.0
On Sunday, June 23, 2013 07:20:50 PM Nico Kadel-Garcia wrote: > On Sun, Jun 23, 2013 at 3:56 PM, Alexey Neyman wrote: > > Hi, > > > > > > > > I've tried upgrading the client to SVN 1.8, and now see some strange merge > > errors while reintegrating the branch. According to > > Did you delete the old RHEL 1.6.11 RPM, to avoid libraries lying > around and confusing you? Or binaries turning up in your "commit" > scripts due to inherited "PATH" that does not include the new version? Sorry for being unclear: I only upgraded the CLIENT, not the server. The server still has 1.6.11. And yes, sorry for stating the obvious, the 1.7.10 subversion RPM was removed when I installed 1.8.0. > To avoid the fun and games, I urge you to grab and test my newly > minted Subversion 1.8.0 RPM building tools, at: > > https://github.com/nkadel/subversion-1.8.x-srpm Are you calling RPMs provided by WanDisco's "fun and games"? I think Subversion developers employed by WanDisco might be somewhat insulted by such judgment. Is there something wrong with RPMs from WanDisco that I must, in your opinion, switch to your version? > I've also bundled a libserf SRPM building toolkit at: > > https://github.com/nkadel/libserf-1.2.x-srpm Thanks. I thought that my original email indicated that we're using svn:// protocol and thus serf is out of the equation. In other words: do you have anything relevant to say regarding the reported issue, rather than advertising your work? Regards, Alexey. > > I've been testing out the 1.8.x features, it's suitable for pushing to > Rpmforge. EPEL would take a lot more work because there are key > libraries, like "SQLite", that have to be compiled locally even for > RHEL 6. And Fedora 19, which is coming out, does not use SysV init > scripts at all, it uses system, so the "svnserve" init script will be > rejected out of hand. > > > http://subversion.apache.org/docs/release-notes/1.8.html#auto-reintegrate > > > > > > > > the --reintegrate option is now deprecated, its use is discouraged and SVN > > should be able to figure that out automatically. However, when I tried a > > plain "svn merge", it gave me the following error: > > > > > > > > [aneyman@build2 trunk]$ svn merge ^/MERGE-PATH > > > > svn: E210002: Unable to connect to a repository at URL 'svn://MERGE-URL' > > > > svn: E210002: Network connection closed unexpectedly > > > > > > > > Strangely, 'svn merge --reintegrate' worked fine. > > > > > > > > We are running 1.6.11 on the server (stock RedHat RPM, "1.6.11-2.el6_1.4" > > version). I installed SVN 1.8.0 RPM from WanDisco ("1.8.0-1") on the > > client. > > > > > > > > Any clues/suggestions as to how to debug this further? > > > > > > > > Regards, > > > > Alexey.
Re: Merge error with SVN 1.8.0
[copying dev@ because I found what the issue is] Hi, Did some further investigation and it turns out that SVN1.8 client creates more connections to the server when performing 'svn merge' - exceeding the xinetd's default number of connections per source (10) and indeed, closing the connection on an unsuspecting client. After increasing the number of connections per source to unlimited, the merge went through. Here are some statistics: SVN 1.7, merge --reintegrate 13 connections total, 5 concurrent connections maximum SVN 1.8, merge 18 connections total, 11 concurrent connections maximum SVN 1.8, merge --reintegrate 5 connections total, 3 concurrent connections maximum So, it looks like the new code for automatic detection of "reintegration merges" in 1.8 spawns a bunch of additional connections. So, the question is - what is the maximum number of connections that a client can create to a server? Does it depend on the size of the change? Size of the svn:mergeinfo? I am not comfortable leaving the server configuration at "unlimited", seeing that xinetd limit is a safety net against runaway client bringing down the server. Regards, Alexey. On Sunday, June 23, 2013 12:56:27 PM Alexey Neyman wrote: Hi, I've tried upgrading the client to SVN 1.8, and now see some strange merge errors while reintegrating the branch. According to http://subversion.apache.org/docs/release-notes/1.8.html#auto-reintegrate the --reintegrate option is now deprecated, its use is discouraged and SVN should be able to figure that out automatically. However, when I tried a plain "svn merge", it gave me the following error: [aneyman@build2 trunk]$ svn merge ^/MERGE-PATH svn: E210002: Unable to connect to a repository at URL 'svn://MERGE-URL' svn: E210002: Network connection closed unexpectedly Strangely, 'svn merge --reintegrate' worked fine. We are running 1.6.11 on the server (stock RedHat RPM, "1.6.11-2.el6_1.4" version). I installed SVN 1.8.0 RPM from WanDisco ("1.8.0-1") on the client. Any clues/suggestions as to how to debug this further? Regards, Alexey.
Re: Merge error with SVN 1.8.0
On Saturday, June 29, 2013 01:18:16 PM Stefan Sperling wrote: > On Fri, Jun 28, 2013 at 03:36:38PM -0700, Alexey Neyman wrote: > > [copying dev@ because I found what the issue is] > > > > Hi, > > > > Did some further investigation and it turns out that SVN1.8 client creates > > more connections to the server when performing 'svn merge' - exceeding > > the xinetd's default number of connections per source (10) and indeed, > > closing the connection on an unsuspecting client. After increasing the > > number of connections per source to unlimited, the merge went through. > > > > Here are some statistics: > > > > SVN 1.7, merge --reintegrate > > 13 connections total, 5 concurrent connections maximum > > > > SVN 1.8, merge > > 18 connections total, 11 concurrent connections maximum > > > > SVN 1.8, merge --reintegrate > > 5 connections total, 3 concurrent connections maximum > > > > So, it looks like the new code for automatic detection of "reintegration > > merges" in 1.8 spawns a bunch of additional connections. So, the question > > is - what is the maximum number of connections that a client can create > > to a server? Does it depend on the size of the change? Size of the > > svn:mergeinfo? > > > > I am not comfortable leaving the server configuration at "unlimited", > > seeing that xinetd limit is a safety net against runaway client bringing > > down the server. > I'm not entirely sure how to estimate the largest number > of connections made by 'svn merge'. > > Note that the number of connections also depends on the amount of > svn:externals involved in checkouts and updates. One new connection > is opened for each external. > > You might be interested in Ivan's RA session reuse patch: > http://svn.haxx.se/dev/archive-2013-06/0292.shtml > http://subversion.tigris.org/issues/show_bug.cgi?id=3763 > This will probably be in 1.9, so it won't help you now. But I still > thought it was worth mentioning since it will address your problem > in the long term. Sorry for a late response. The checked out WC I did a merge in did not have any externals, but it had 8 svn:mergeinfo entries, if it matters. I think at the very least this issue needs to be reflected in the "SVN book". Right now, the section "svnserve via xinetd" does not mention it, nor does the example configuration specify an unlimited number of connections per source. Given that xinentd's default is 10 connections per source IP, it is fairly easy to go over that limit. As to session reuse: am I right that it is going to avoid closing connections and reuse it instead of opening a new one? In that case, it does not solve an issue with concurrent connections (as they're still active, they can't be reused). Regards, Alexey.
Re: Merge error with SVN 1.8.0
On Sunday, June 30, 2013 04:33:45 PM Branko Čibej wrote: > On 24.06.2013 01:38, Alexey Neyman wrote: > > Are you calling RPMs provided by WanDisco's "fun and games"? I think > > Subversion developers employed by WanDisco might be somewhat insulted > > by such judgment. > > Hmm I'd sooner be insulted by someone anticipating my reaction like > that. :) > > Nobody's perfect. If you find a bug in WANdisco's binary packages, I for > one would definitely like to know about it. Sorry for insulting you then :) My point was that the problem is most likely not a *packaging* issue, which was confirmed in my next message (the issue is due to 1.8.x making more connections from client to server when doing 'svn merge'). Regards, Alexey.
svnmucc assertion failure
Hi, I've hit the following assertion in svnmucc: svnmucc: subversion/libsvn_subr/dirent_uri.c:1373: svn_uri_get_longest_ancestor: Assertion `svn_uri_is_canonical(uri1, ((void *)0))' failed. Aborted (core dumped) The assertion was triggered when svnmucc was supplied with a script file that had inconsistent URL specifications (it was cut-and-pasted from two different scripts, one of which referred to SVN server by IP and the other by its domain name). It can be easily reproduced with the following script, for example (no repository setup is needed, as this assertion is triggered even before svnmucc contacts the repository): mkdir svn://localhost/svn/branches/try cp 1234 svn://127.0.0.1/svn/trunk/foo svn://localhost/svn/branches/try/foo I am not sure if it is a valid use case, but at the very least, the error message should probably be me a bit more user-friendly than an assertion. Patch erring out gracefully attached. Regards, Alexey. Index: subversion/svnmucc/svnmucc.c === --- subversion/svnmucc/svnmucc.c (revision 1508998) +++ subversion/svnmucc/svnmucc.c (working copy) @@ -1433,7 +1433,14 @@ if (! anchor) anchor = url; else -anchor = svn_uri_get_longest_ancestor(anchor, url, pool); +{ + anchor = svn_uri_get_longest_ancestor(anchor, url, pool); + if (!anchor[0]) +handle_error(svn_error_createf(SVN_ERR_INCORRECT_PARAMS, NULL, + "URLs in the action list do not " + "share a common ancestor\n"), + pool); +} if ((++i == action_args->nelts) && (j + 1 < num_url_args)) insufficient(pool);
authz via properties?
Hi all, We are actively using authz path-based authentication rules: due to some legal requirements, some parts of our product source code are not accessible to a part of the developer team. Currently authz does not support wildcards (there is an issue about that [1] discussed since 2006). Because of this, each time a branch is created, authz rules have to be copied and modified for the new branch. This leads to a proliferation of authz rules; our authz is currently about 2000 lines and growing. I am currently implementing a post-commit script so that we would be able to record authz rules on files/directories, and authz would be appended with new rules every time these files/directories are copied. First, I am wondering how well such 'authz' approach would scale. Has anyone run scalability tests on authz? Second, I thought that if I am using properties to track authz-controlled files, SVN server would probably do that more effectively than a post-commit script. As an added value, property-based authz would allow versioning in path-based auth configuration that current mechanism does not allow. E.g., currently one could either configure path /foo as either R/O, R/W or unaccessible to user U; it is not possible to configure the path to be unaccessible before/after a certain revision. Thoughts? Ideas? Regards, Alexey. [1] http://subversion.tigris.org/issues/show_bug.cgi?id=2662
Re: authz via properties?
[Restored CC of the mailing list] On Thursday, October 17, 2013 03:24:45 PM Mark Phippard wrote: On Thu, Oct 17, 2013 at 3:00 PM, Alexey Neyman wrote: We are actively using authz path-based authentication rules: due to some legalrequirements, some parts of our product source code are not accessible to apart of the developer team. Currently authz does not support wildcards (thereis an issue about that [1] discussed since 2006). Because of this, each time abranch is created, authz rules have to be copied and modified for the newbranch. This leads to a proliferation of authz rules; our authz is currently about2000 lines and growing. I am currently implementing a post-commit script sothat we would be able to record authz rules on files/directories, and authzwould be appended with new rules every time these files/directories arecopied. CollabNet TeamForge supports wildcard rules: http://blogs.collab.net/teamforge/wildcard-access-control-and-path-based-permissions-in-teamforge[2] Interesting. How did they deal with the concern raised in issue #2662 (i.e., the need to walk the tree below a certain path to check if any of the other rules apply to any descendant path)? First, I am wondering how well such 'authz' approach would scale. Has anyonerun scalability tests on authz? So your question is whether at a certain size does it slow down? I recall in the past it being said that it was stored in a hash so performance should not vary. But there has to be a parsing slow down and possible memory bloat. That said, I have heard of files in the hundred thousands for lines. Yes, that was the question. Note that you can also have files per repository. We do not want to split the repository unless absolutely necessary, as that would break the atomicity of commits for features touching both restricted and unrestricted parts of the repository. Instead, I think, it would be very handy if the access rights were copied along with the file/directory on which they are specified. Second, I thought that if I am using properties to track authz-controlledfiles, SVN server would probably do that more effectively than a post-commitscript. As an added value, property-based authz would allow versioning inpath-based auth configuration that current mechanism does not allow. E.g.,currently one could either configure path /foo as either R/O, R/W orunaccessible to user U; it is not possible to configure the path to beunaccessible before/after a certain revision. Someone could always contribute it. I do not think it would scale well but if it were optional then you could make the decision for yourself. Authz rules are expensive to apply. If SVN had to do additional repository I/O to check for and fetch properties it would only get worse. I'll probably have a stab at it. One of the goals of this post was to check if there are any objections to such feature that would make such development worthless /ab initio/. Regards, Alexey. [1] mailto:sti...@att.net [2] http://blogs.collab.net/teamforge/wildcard-access-control-and-path-based-permissions-in-teamforge
Re: authz via properties?
On Friday, October 18, 2013 02:46:39 AM Branko Čibej wrote: On 17.10.2013 20:00, Alexey Neyman wrote: Hi all, We are actively using authz path-based authentication rules: due to some legal requirements, some parts of our product source code are not accessible to a part of the developer team. Currently authz does not support wildcards (there is an issue about that [1] discussed since 2006). Because of this, each time a branch is created, authz rules have to be copied and modified for the new branch. This leads to a proliferation of authz rules; our authz is currently about 2000 lines and growing. I am currently implementing a post-commit script so that we would be able to record authz rules on files/directories, and authz would be appended with new rules every time these files/directories are copied. First, I am wondering how well such 'authz' approach would scale. Has anyone run scalability tests on authz? Second, I thought that if I am using properties to track authz-controlled files, SVN server would probably do that more effectively than a post-commit script. As an added value, property-based authz would allow versioning in path-based auth configuration that current mechanism does not allow. E.g., currently one could either configure path /foo as either R/O, R/W or unaccessible to user U; it is not possible to configure the path to be unaccessible before/after a certain revision. Thoughts? Ideas? Properties are not suitable for storing ACLs because they are immutable; i.e., you cannot change properties on committed files and directories. You need a different kind of structure, one that the Subversion repository does not have yet. Well, technically you can dump & reload... But that's hardly maintainable, I agree. Are those ACLs you're describing going to be version-specific? In other words, will it be possible to specify that /foo/bar@2345 is r/w for user harry, but not accessible starting with revision 2346? Thanks for a pointer to that 1.8 feature, I forgot about that. That might make my task a bit easier. Regards,Alexey. -- Branko Čibej | *Director of Subversion* WANdisco // /Non-Stop Data/ e. br...@wandisco.com[1] [1] mailto:br...@wandisco.com
SVN 1.8.8: assertion failed (move_src_op_root_relpath != NULL && move_dst_op_root_relpath != NULL)
Hi Subversion developers, Encountered the following assertion while trying to resolve a tree conflict: svn: E235000: In file 'subversion/libsvn_wc/wc_db_update_move.c' line 2462: assertion failed (move_src_op_root_relpath != NULL && move_dst_op_root_relpath != NULL) ./svn-bug.sh: line 33: 30387 Aborted (core dumped) svn resolved -R dir2 Environment: RHEL6, x86_64, Subversion 1.8.8 RPM from WanDisco. svn, version 1.8.8 (r1568071) compiled Feb 17 2014, 18:16:33 on x86_64-unknown-linux-gnu Copyright (C) 2013 The Apache Software Foundation. This software consists of contributions made by many people; see the NOTICE file for more information. Subversion is open source software, see http://subversion.apache.org/ The following repository access (RA) modules are available: * ra_svn : Module for accessing a repository using the svn network protocol. - with Cyrus SASL authentication - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - using serf 1.3.2 - handles 'http' scheme - handles 'https' scheme Reproduction script: [[[ #!/bin/bash -x repo=/tmp/repo url=file://$repo wc1=/tmp/wc1 wc2=/tmp/wc2 # Create initial set-up rm -rf $repo $wc1 $wc2 svnadmin create $repo svn co $url $wc1 cd $wc1 svn mkdir dir1 dir2 echo hi > dir1/file1 echo lo > dir2/file2 svn add dir1/file1 dir2/file2 svn ci -m "Create initial layout" svn up # Prepare local changes in WC #1 svn mv dir2/file2 dir1/file2 svn rm --force dir2 # Make conflicting changes in WC #2 svn co $url $wc2 cd $wc2 echo more >> dir2/file2 svn ci -m "Modify dir2" # Try to merge in WC #1 cd $wc1 svn up --accept=mc svn resolved -R dir2 ]]] Regards, Alexey.
Re: SVN 1.8.8: assertion failed (move_src_op_root_relpath != NULL && move_dst_op_root_relpath != NULL)
Just verified the issue is present on trunk as well (r1587745): subversion/libsvn_wc/wc_db_update_move.c:2493: (apr_err=SVN_ERR_ASSERTION_FAIL) svn: E235000: In file 'subversion/libsvn_wc/wc_db_update_move.c' line 2493: assertion failed (move_src_op_root_relpath != NULL && move_dst_op_root_relpath != NULL) ./svn-bug.sh: line 33: 30996 Aborted (core dumped) svn resolved -R dir2 Regards, Alexey. On Tuesday, April 15, 2014 03:52:48 pm Alexey Neyman wrote: > Hi Subversion developers, > > Encountered the following assertion while trying to resolve a tree > conflict: > > svn: E235000: In file 'subversion/libsvn_wc/wc_db_update_move.c' line 2462: > assertion failed (move_src_op_root_relpath != NULL && > move_dst_op_root_relpath != NULL) > ./svn-bug.sh: line 33: 30387 Aborted (core dumped) svn > resolved -R dir2 > > Environment: RHEL6, x86_64, Subversion 1.8.8 RPM from WanDisco. > > svn, version 1.8.8 (r1568071) >compiled Feb 17 2014, 18:16:33 on x86_64-unknown-linux-gnu > > Copyright (C) 2013 The Apache Software Foundation. > This software consists of contributions made by many people; > see the NOTICE file for more information. > Subversion is open source software, see http://subversion.apache.org/ > > The following repository access (RA) modules are available: > > * ra_svn : Module for accessing a repository using the svn network > protocol. - with Cyrus SASL authentication > - handles 'svn' scheme > * ra_local : Module for accessing a repository on local disk. > - handles 'file' scheme > * ra_serf : Module for accessing a repository via WebDAV protocol using > serf. - using serf 1.3.2 > - handles 'http' scheme > - handles 'https' scheme > > > Reproduction script: > [[[ > #!/bin/bash -x > > repo=/tmp/repo > url=file://$repo > wc1=/tmp/wc1 > wc2=/tmp/wc2 > > # Create initial set-up > rm -rf $repo $wc1 $wc2 > svnadmin create $repo > svn co $url $wc1 > cd $wc1 > svn mkdir dir1 dir2 > echo hi > dir1/file1 > echo lo > dir2/file2 > svn add dir1/file1 dir2/file2 > svn ci -m "Create initial layout" > svn up > > # Prepare local changes in WC #1 > svn mv dir2/file2 dir1/file2 > svn rm --force dir2 > > # Make conflicting changes in WC #2 > svn co $url $wc2 > cd $wc2 > echo more >> dir2/file2 > svn ci -m "Modify dir2" > > # Try to merge in WC #1 > cd $wc1 > svn up --accept=mc > svn resolved -R dir2 > ]]] > > Regards, > Alexey.
Re: Regression in bindings? 1.7/1.8 vs 1.6
[CC'ing users@] On Sunday, August 10, 2014 08:35:44 PM Ben Reser wrote: > > Also, I think it would be a good idea to have the transaction-modifying > > functions return an error once the transaction reached the stage of > > pre-commit hooks from functions like svn_fs_change_node_prop() - to avoid > > getting one's hopes high because it works in the current release. > > Something like SVN_TXN_READONLY. What do you think? > > Keeping in mind that Subversion doesn't have a central process that owns the > file system, it'd have to be something we write out to the transaction. > I'm not sure how practical that is given our current formats. But yes this > might not be a bad idea since it's not something we intend to allow. I > haven't done too much digging but that might even be what we're doing. That's what I meant - the functions like svn_fs_change_node_prop() will return an error status, not that some "owner process" will report an error to the system log. Is there a way for these functions could check whether the FS root passed as the first argument belongs to a transaction that is currently executing the pre-commit hook? > > But back to use case: I am thinking about alternative approaches to doing > > such auto-updates of properties and/or other content. I assume that it is > > not possible to create a transaction B based on a transaction A in the > > pre-commit hook (so that when transaction A becomes a revision, > > transaction B uses that new revision as a base), is it? > > > > It seems that the only supported way to do that would be to schedule the > > "update tasks" to be done in the pre-commit script, but actually execute > > them in a new transaction. Hence, another question - is a post-commit > > hook allowed to create and commit a transaction, or does it have to be > > deferred until after the post-commit hook finishes? > > Perhaps it would be better to tell us what you're trying to do rather than > talking about how you're trying to do it. I can't think of a good reason > why you need to be modifying properties on files like this. I described this in the previous email: I want to have one file, /project/trunk/include/version.h, reflect the last revision that touched any file in the project. For that purpose, pre-commit hook (on a server currently using SVN 1.6) checks if any file under /project/trunk is modified and if it's the case, modifies a property on the /project/trunk/include/version.h. This in turn causes $Revision$ in to reflect the last revision when ANY file in the project was modified, not when the itself was modified. There is another case where our current scripts modified the node properties during pre- commit: we use a modified version of FreeBSD's verify.py, and among other checks, it uses heuristics to determine whether a file is binary or text. The FreeBSD's version checks that every time, while we tried to cache the result of that heuristicss in a file's property. Am I missing some obvious way to do these tasks? > Additionally, I'd point out that this whole thread should probably be > happening on users@subversion.apache.org. I don't think this is a bug. > You may also get more responses to your questions there, since you're > hitting a much broader audience that largely consists of admins instead of > just developers. I'd guess some of those admins have dealt with similar > problems. By contracts us developers don't typically administer > repositories, in fact since joining the ASF we don't even admin our own > repository. CC'ed users@; feel free to drop dev@ when replying. Regards, Alexey.
unversioned properties: size limitations?
Hi SVN users/developers, Is there a limitation in size on the property value that can be set? Any scalability traps to be aware of (i.e. non-linear increase in time due to increase in size of the property value)? I tried a 4Mb property, seems to work fine... Thanks, Alexey.
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 06:59:20 AM Branko Čibej wrote: > On 12.08.2014 03:31, Alexey Neyman wrote: > > Hi SVN users/developers, > > > > Is there a limitation in size on the property value that can be set? > > Any scalability traps to be aware of (i.e. non-linear increase in time > > due to increase in size of the property value)? I tried a 4Mb > > property, seems to work fine... > > One thing to be aware of is that properties were never designed to be > large. Property values are always transmitted in full text between > client and server (i.e., they're not compressed); they're stored in full > text in the repository (not deltified the way file contents are). So the > more large properties you have, and the more often you modify them, the > less efficient your repository will be, in terms of storage requirements > and network bandwidth. > > So while you should be able to store a 2 gig property value, I really, > really recommend not to do that. I thought of having pre- and post-commit hooks communicate using a *revision* property: pre-commit hook would set a revision property with the list of files and actions to be performed on them, and the post-commit hook will perform these actions by committing a new revision (instead of modifying a transaction by pre-commit hook). Thus a more specific question - when are arbitrary *revision* properties sent from the client to the server? Obviously, svn:* properties are used by various SVN commands; but am I right to assume that non-standard revision properties are sent only for the 'svn pg -- revprop' command? That said, I expect the property value to be much less than 2Gb. So far, the largest commit we've had was ~2 files - with ~150 characters per path, that would be about 3Mb for the property value. Regards, Alexey.
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 07:44:03 AM Branko Čibej wrote: > On 12.08.2014 07:16, Alexey Neyman wrote: > > On Tuesday, August 12, 2014 06:59:20 AM Branko Čibej wrote: > > > On 12.08.2014 03:31, Alexey Neyman wrote: > > > > Hi SVN users/developers, > > > > > > > > > > > > > > > > Is there a limitation in size on the property value that can be set? > > > > > > > > Any scalability traps to be aware of (i.e. non-linear increase in time > > > > > > > > due to increase in size of the property value)? I tried a 4Mb > > > > > > > > property, seems to work fine... > > > > > > One thing to be aware of is that properties were never designed to be > > > > > > large. Property values are always transmitted in full text between > > > > > > client and server (i.e., they're not compressed); they're stored in full > > > > > > text in the repository (not deltified the way file contents are). So the > > > > > > more large properties you have, and the more often you modify them, the > > > > > > less efficient your repository will be, in terms of storage requirements > > > > > > and network bandwidth. > > > > > > > > > > > > So while you should be able to store a 2 gig property value, I really, > > > > > > really recommend not to do that. > > > > I thought of having pre- and post-commit hooks communicate using a > > *revision* property: pre-commit hook would set a revision property > > with the list of files and actions to be performed on them, and the > > post-commit hook will perform these actions by committing a new > > revision (instead of modifying a transaction by pre-commit hook). > > > > > > > > Thus a more specific question - when are arbitrary *revision* > > properties sent from the client to the server? Obviously, svn:* > > properties are used by various SVN commands; but am I right to assume > > that non-standard revision properties are sent only for the 'svn pg > > --revprop' command? > > > > > > > > That said, I expect the property value to be much less than 2Gb. So > > far, the largest commit we've had was ~2 files - with ~150 > > characters per path, that would be about 3Mb for the property value. > > Sure, you'll only transmit revprops with propget --revprop and propset > --revprop. I'm not sure what the implications are of storing large > values in revprops, these are handled a bit differently than versioned > properties on the server. > > And of course, revision properties are not versioned. > > I'm still not sure what you're trying to achieve, though. "Communication > between pre- and post-commit hooks" doesn't describe the problem, it > describes a solution, and there are of course other ways for hooks to > communicate that do not involve the repository. I've mentioned this in the other thread where you also responded. There are two problems that are currently (we're using 1.6) solved by modifying the transaction in the pre-commit hook: 1. We have a header that needs to reflect the last modification date of *any* file in the project. Currently, pre-commit script modifies a property in each commit which touches any file in /project/trunk. 2. We have a few checks in pre-commit that are performed on text but not on binary files, and (unless it the type is set explicitly), the text is distinguished from binaries using simple heuristics. To avoid running this heuristics over and over, the result is saved into a property on that file. So, to avoid modifying the transaction by pre-commit (that no longer reliably works in 1.7 and 1.8), I am changing the pre-commit to list the 'tasks' to be performed by the post- commit - which will check in a new revision. I don't want to involve an out-of-repository storage for that list of tasks unless absolutely necessary - hence, revision property looks like the perfect place to store the "follow-up tasks" for a particular revision. > Also I find your approach less than robust: > > * There's no guarantee that the post-commit hook will ever run, so > it's a bad idea to rely on it for anything that's critical to your > workflow. Can you please elaborate on this? I thought that if a transaction was promoted to a revision, the post-commit hook is always run. I understand that post-commit may fail and this failure will not roll back a revision. But when is it not run at all? PS. I know that there's a
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 08:33:06 AM Branko Čibej wrote: > On 12.08.2014 08:17, Alexey Neyman wrote: > > On Tuesday, August 12, 2014 07:44:03 AM Branko Čibej wrote: > > > I'm still not sure what you're trying to achieve, though. "Communication > > > > > > between pre- and post-commit hooks" doesn't describe the problem, it > > > > > > describes a solution, and there are of course other ways for hooks to > > > > > > communicate that do not involve the repository. > > > > I've mentioned this in the other thread where you also responded. > > There are two problems that are currently (we're using 1.6) solved by > > modifying the transaction in the pre-commit hook: > > > > > > > > 1. We have a header that needs to reflect the last > > modification date of *any* file in the project. Currently, pre-commit > > script modifies a property in each commit which touches any file in > > /project/trunk. > > So why do you need the last-changed revision of the project directory > stored in the header file? Are you distributing sources directly from > the repository, or are you using that number to identify builds? In > either case, there are better ways to do that. Sometimes, yes. Not directly from a repository but engineers sometimes release semi- official builds to customers. I understand that is less than ideal way (not dealing with mixed-revision WCs, etc.). So, I am curious what are those better ways you mention? > > 2. We have a few checks in pre-commit that are performed on text but > > not on binary files, and (unless it the type is set explicitly), the > > text is distinguished from binaries using simple heuristics. To avoid > > running this heuristics over and over, the result is saved into a > > property on that file. > > I'd just like to point out that Subversion already has heuristics to > identify binary files. If you add a binary file to the working copy, it > will automagically get the svn:mime-type set to > application/octet-stream. There are only a few MIME types that > Subversion does not consider binary. Of course, your heuristics might be > different from Subversion's, but have you considered just looking at the > mime-type property? Yes, I've considered it, there are a few drawbacks: - First, the repository was imported from CVS and many binary files were not marked with the svn:mime-type properly set at the conversion time (most likely, because the oridinal CVS file didn't have -kb keyword expansion set - but that's now history). This could be solved by going over all the project files once and setting svn:mime-type explicitly, though. - Second, some of the MIME types are text even though the MIME type does not start with 'text/*'. "image/svg", "application/x-sh" and "application/xml" are a few most obvious examples that come to mind. I don't know how to come up with an exhaustive list of such "text-despite-MIME-type" types. Because of these I decided to use a separate property. > > So, to avoid modifying the transaction by pre-commit (that no longer > > reliably works in 1.7 and 1.8), > > Correction: that never reliably worked, because we never promised it > would work. We haven't noticed any issues with 1.4, 1.5 and 1.6 :) > > I am changing the pre-commit to list the 'tasks' to be performed by > > the post-commit - which will check in a new revision. I don't want to > > involve an out-of-repository storage for that list of tasks unless > > absolutely necessary - hence, revision property looks like the perfect > > place to store the "follow-up tasks" for a particular revision. > > > > > Also I find your approach less than robust: > > > > > > > > > > > > * There's no guarantee that the post-commit hook will ever run, so > > > > > > it's a bad idea to rely on it for anything that's critical to your > > > > > > workflow. > > > > Can you please elaborate on this? I thought that if a transaction was > > promoted to a revision, the post-commit hook is always run. I > > understand that post-commit may fail and this failure will not roll > > back a revision. But when is it not run at all? > > It's entirely possible that the commit succeeds, but for any number of > reasons -- power failure, full disk, simple misconfiguration -- the > post-commit hook doesn't run. Subversion does not make any guarantees > there other than making a best effort to run it. Well, these are force majeure sort of issues. For the u
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 09:43:39 AM Branko Čibej wrote: > On 12.08.2014 09:26, Alexey Neyman wrote: > > On Tuesday, August 12, 2014 08:33:06 AM Branko Čibej wrote: > > > So why do you need the last-changed revision of the project directory > > > stored in the header file? Are you distributing sources directly from > > > the repository, or are you using that number to identify builds? In > > > either case, there are better ways to do that. > > > > Sometimes, yes. Not directly from a repository but engineers sometimes > > release semi-official builds to customers. > > > > I understand that is less than ideal way (not dealing with > > mixed-revision WCs, etc.). So, I am curious what are those better ways > > you mention? > > I'm sort of assuming a defined build (or packaging) workflow. If you > have engineers mailing binaries from their mixed-revision development > working copies to customers, all bets are off, since they might have > locally modified your version property. > > One solution is to run "svnversion" as part of the package/build and > include the result in the build artefacts. Can't go into any more detail > than that, since I have no ides what your build system looks like. This > won't always perfectly identify the configuration you're building, but > at least it will give some indication of the state of the working copy, > and whether (oh horror) it includes local modifications. 1. svnversion reports the revision of the check-out, not the revision of the last modification. 2. svnversion does not provide the $Date$ information. 3. svnversion won't report anything on 'svn export'-ed sources. All that would be handled if there were a way to modify a transaction being committed :) > > - Second, some of the MIME types are text even though the MIME type > > does not start with 'text/*'. "image/svg", "application/x-sh" and > > "application/xml" are a few most obvious examples that come to mind. I > > don't know how to come up with an exhaustive list of such > > "text-despite-MIME-type" types. > > Heh, tell me about it. Subversion has the same problem, and we still > haven't found a satisfactory solution for it. Well, if Subversion set an svn:binary to true/false just to report the heuristics, in addition to svn:mime-type, that would've helped somewhat. But Subversion heuristics only runs on newly added files. > So you really only have to add this property on added files, right? Have > you considered using auto-props? They don't currently give you exactly > the knob you need, because we don't have an auto-prop setting for "all > binary" or "all text" files, but maybe that could solve your problem. No, it is also set on existing files as they're modified. > Especially now that, with 1.8, you can define auto-prop settings in the > repository and no longer need local configuration on every client. > (Caveat: you do need 1.8 clients.) Now you tell me about that. I am still trying to weed out 1.6 clients (which break horribly by nested 'svn mv')- since CentOS does not update the base version of the packages during the release lifetime, CentOS 6.x is stuck with Subversion 1.6. Thanks to WanDisco for providing the CentOS RPMs for Subversion 1.7/1.8 :) It would be nice if there were a 'start-access' script that is run on any access to the repository - that can be used to block such older clients from even accessing the repository. Speaking of 'wish' items: - How hard would be to implement client-side hooks? I know TortoiseSVN has them, but they're local - and to be really useful, they need to be propagated from the repository. And it would be nice to have them supported by the default commandline client. If they were supported, the tasks like the ones described above could be relegated to the client side, and the server-side hooks could then just reject the commit if the client failed to run the script. - How hard would be to allow modifications to a transaction by pre-commit? If it is just the issue of client-side caches going stale - can a "shadow transaction" be created that records all the modifications by the pre-commit script and then sends it back to the client to apply to the WC? Regards, Alexey.
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 11:49:39 am Ben Reser wrote: > I intended to reply to your other thread given roughly the same svnversion > advice that Branko has already given you. > > On 8/12/14 9:09 AM, Alexey Neyman wrote: > > 1. svnversion reports the revision of the check-out, not the revision of > > the last modification. > > Use one of the alternatives that behaves the way you want. To save > retyping info I've already provided elsewhere see this: > https://stackoverflow.com/questions/21447373/how-to-get-correct-svn-version > -number-on-unix/21448972#21448972 > > For reference SubWCRev is something that comes with TortoiseSVN: > http://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-subwcrev.html Actually, it's even easier than that. I somehow missed the -c option to svnversion. > > 2. svnversion does not provide the $Date$ information. > > Patches welcome as always. If you don't want to write the patch you can > always ask for it. I think I can write such patch myself :) > > 3. svnversion won't report anything on 'svn export'-ed sources. > > Your properties won't be available either. Properties won't, but expanded keywords will be present in 'svn export'. $Revision$ will change in response to a property change. Please consider these questions closed. For the 'last changed rev/date' I'll modify the 'svnversion -c' and will submit a patch to support reporting the dates as well (hopefully, it can then be applied to 1.8.x branch as well) and for the caching the results of file analysis, I'll use a post-commit script (since not performing that action does not compromise the integrity of the project in the repository). Regards. Alexey.
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 11:53:09 AM Ben Reser wrote: > On 8/12/14 9:31 AM, Branko Čibej wrote: > > For a start, this would require a major change in the wire protocol, where > > the server would, as a response to a successful commit, report any > > additional "magic" changes to the client. The problem with this is that > > it is error prone; the response may never arrive, for any number of > > reasons. Therefore, the client could not mark commtited items up-to-date > > until and unless it received the response. Since at least the DAV > > protocol is stateless, this implies all sorts of complications and the > > introduction of intermediate states in the working copy. > > > > In short: yes, it'd be hard. > > Ignoring all that... There's a better reason why it won't happen. > > Subversion clients before whatever version we add it to won't support it. > Which leaves those clients with stale caches. You'd have to disallow it > with older clients or just ignore the stale cache problem. I frankly do > not see the community accepting a change that ignores such a huge problem > with old clients. Isn't that the same kind of change that happened with version 1.5 and mergeinfo? If one wanted to use mergeinfo, one had to have 1.5+ clients. A capability reporting was added, and a server can check that only mergeinfo-capable clients can start a commit. Same here, if a repository administrator wants to have pre-commit scripts that modify a transaction, he'd better check the clients' ability to handle a change to be applied to WC in server's response. Regards, Alexey.
Re: unversioned properties: size limitations?
On Tuesday, August 12, 2014 10:18:48 PM Ben Reser wrote: > On 8/12/14 7:02 PM, Alexey Neyman wrote: > > Isn't that the same kind of change that happened with version 1.5 and > > mergeinfo? If one wanted to use mergeinfo, one had to have 1.5+ clients. A > > capability reporting was added, and a server can check that only > > mergeinfo-capable clients can start a commit. Same here, if a repository > > administrator wants to have pre-commit scripts that modify a transaction, > > he'd better check the clients' ability to handle a change to be applied > > to WC in server's response. > > If a server admin changes a transaction and the commit came from an old > client then the working copy is potentially broken. If an old client > doesn't know about mergeinfo it doesn't have access to that new feature. > New clients lose awareness of any merges committed by the old clients but > neither side is really broken. You just may be inconvenienced when > merging. I think there's a huge difference there. I think this thread quickly turns into a discussion how to roll out a feature that's not going to be implemented in the foreseeable future. I still think there would be feasible ways: Leaving it up to repository admin's configuration of start-commit - which is no worse than the current behavior if that repository's pre-commit modifies a transaction. OR Rejecting modifications to a transaction if the transaction was originated by non-aware client. OR Rejecting to promote the transaction into the revision if it came from non-aware client and was modified by pre-commit. It is pointless to discuss which if these approaches is better and/or acceptable, though, unless there is a decision to implement this feature. Regards, Alexey
Re: Creating a pre-commit hook for copy operations
Another (and possibly easier) way is to use fs.paths_changed2 - the entries in that hash have copyfrom_path/copyfrom_rev fields indicating the source of the copy. Regards, Alexey. On Friday, August 15, 2014 11:24:18 am Keath Milligan wrote: > Thanks, Eric. svnlook gave me the clue I needed. I look at its source and > found that I had been on the right track using ChangeCollector and replay2, > but there was an error in my code. Once I got that fixed, my hook worked. > > On Fri, Aug 15, 2014 at 12:59 PM, Eric Johnson wrote: > > I had a similar problem of needing to look for copies. Rather than deal > > with the complexity of the API, I simply relied on the "svnlook" command > > output. > > > > Specifically, I used "svnlook --transaction ${tx_id} changed > > --copy-info", and parsed the result. > > > > Eric > > > > > > On Fri, Aug 15, 2014 at 7:15 AM, Keath Milligan > > > > wrote: > >> We're using Subversion 1.8.3 and need to create a pre-commit hook that > >> checks copy operations (specifically branch creation) for certain > >> conditions. Specifically, we want to be able to prevent users from > >> creating branches in the wrong locations and similar errors. > >> > >> To accomplish this, we need to know the source and destination paths of > >> the copy operation. Our current pre-commit hook (written in Python) uses > >> a delta.Editor subclass to check other things. We tried overriding the > >> add_directory method, which has a copyfrom_path argument -- this would > >> have been perfect, but this argument is always empty. > >> > >> We also tried using the repos.Changecollector and repos.replay2, but > >> this doesn't seem to give us information about the pending commit -- > >> perhaps we are doing it wrong. > >> > >> Any suggestion would be much appreciated. > >> > >> Thanks
Problem with subversion-perl RPMs from WanDisco?
Hi all, I tried to update subversion to 1.8.10 on a freshly installed CentOS 6.5 and noticed that an attempt to use Perl bindings has failed due to an undefined symbol in the libsvn_swig_perl library: svn_swig_pl_get_current_pool $ readelf -Ws /tmp/usr/lib64/libsvn_swig_perl-1.so.0 | grep svn_swig_pl_get_ 79: 0 NOTYPE GLOBAL DEFAULT UND svn_swig_pl_get_current_pool This symbol is defined in the library packaged into 1.9.0-alpha2 RPMs, but not in any 1.8.x/1.7.x RPMs. I have also tried on another machin - it originally had CentOS 6.0 installed + partially upgraded. There, the Subversion Perl bindings from WanDisco's RPM just coredump in Repos::Open - I am not sure if it is the same issue or not. It works with the original CentOS version of subversion-perl (1.6.x). Regards, Alexey.
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 09:45:32 am Philip Martin wrote: > Alexey Neyman writes: > > I tried to update subversion to 1.8.10 on a freshly installed CentOS 6.5 > > and noticed that an attempt to use Perl bindings has failed due to an > > undefined symbol in the libsvn_swig_perl library: > > svn_swig_pl_get_current_pool > > > > $ readelf -Ws /tmp/usr/lib64/libsvn_swig_perl-1.so.0 | grep > > svn_swig_pl_get_ > > > > 79: 0 NOTYPE GLOBAL DEFAULT UND > > > > svn_swig_pl_get_current_pool > > I see that the 1.8 WANdisco package installs some files into > /usr/local/lib64. The file > > /usr/local/lib64/perl5/auto/SVN/_Core/_Core.so > > is the one that defines your missing symbol. Perhaps the problem is > that the way you are invoking Perl means it does not load files from > /usr/local? Rather, I think, it is that the libsvn_swig_perl-1.so.0 does not list the _Core.so as a dependency: $ ldd /usr/lib64/libsvn_swig_perl-1.so linux-vdso.so.1 => (0x7fffea3ff000) libsvn_delta-1.so.0 => /usr/lib64/libsvn_delta-1.so.0 (0x7f31ce70) libsvn_subr-1.so.0 => /usr/lib64/libsvn_subr-1.so.0 (0x7f31ce3fa000) libz.so.1 => /lib64/libz.so.1 (0x7f31ce1cc000) libdl.so.2 => /lib64/libdl.so.2 (0x7f31cdfc8000) libaprutil-1.so.0 => /usr/lib64/libaprutil-1.so.0 (0x7f31cdda4000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x7f31cdb6c000) libexpat.so.1 => /lib64/libexpat.so.1 (0x7f31cd944000) libdb-4.7.so => /usr/lib64/libdb-4.7.so (0x7f31cd5d) libapr-1.so.0 => /usr/lib64/libapr-1.so.0 (0x7f31cd3a3000) libpthread.so.0 => /lib64/libpthread.so.0 (0x7f31cd186000) libc.so.6 => /lib64/libc.so.6 (0x7f31ccdf2000) /lib64/ld-linux-x86-64.so.2 (0x003acd60) libuuid.so.1 => /lib64/libuuid.so.1 (0x7f31ccbed000) libfreebl3.so => /usr/lib64/libfreebl3.so (0x7f31cc976000) If I pre-load the library, though, it fails differently: $ LD_PRELOAD=/usr/local/lib64/perl5/auto/SVN/_Core/_Core.so /tmp/1.pl /repo Can't locate object method "new_default" via package "SVN::Pool" (perhaps you forgot to load "SVN::Pool"?) at /tmp/1.pl line 6. [[[ 1.pl #!/usr/bin/perl use SVN::Repos; die "Usage: 1.pl REPO" unless $#ARGV == 0; my $repo = SVN::Repos::open($ARGV[0]) or die "Cannot open repo " . $ARGV[0]; exit 0; ]]] > You might be able to set LD_DEBUG to diagnose the problem, > something like > > LD_DEBUG=libs perl file.pl Just as I said above, it does not even try to search for _Core.so: $ LD_DEBUG=libs /tmp/1.pl /repo 2>&1 | grep _Core $ As a side question, why does this package install stuff into both /usr/lib and /usr/local/lib? Regards, Alexey. > > (run any executable with LD_DEBUG=help to see other options). > > > This symbol is defined in the library packaged into 1.9.0-alpha2 RPMs, > > but not in any 1.8.x/1.7.x RPMs. > > The handling of the symbol changed in 1.9.
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 11:16:12 am Philip Martin wrote: > Alexey Neyman writes: > > Rather, I think, it is that the libsvn_swig_perl-1.so.0 does not list the > > > _Core.so as a dependency: > I don't think that's the way it works. When Perl uses a module such as > SVN::Repos then Perl itself knows to uses SVN::Core as a dependency. > That causes the shared objects _Repos.so and _Core.so to be loaded. You > should see that _Core.so has libsvn_swig_perl-1.so as a dependency. Apparently it doesn't know that - see below. > > If I pre-load the library, though, it fails differently: > > > > $ LD_PRELOAD=/usr/local/lib64/perl5/auto/SVN/_Core/_Core.so /tmp/1.pl > > /repo Can't locate object method "new_default" via package "SVN::Pool" > > (perhaps you forgot to load "SVN::Pool"?) at /tmp/1.pl line 6. > > I'm not sure LD_PRELOAD is the right way to do it as that might bypass > the full Perl module load mechanism. I think you should try setting > @INC, perhaps via > > use lib '/usr/local/lib64/perl5'; Unfortunately, this does not work. What works (at least getting it past the point of loading the library) is explicit 'use SVN::Core;' before 'use SVN::Repos'. But, I have verified this is an issue with the stock CentOS packages (1.6.11), so it seems to be a generic issue. I removed too much when reducing the test case. So, the question is, should SVN::Repos load SVN::Core automatically? However, it then segfaults in a shared library. [aneyman@localhost ~]$ gdb /usr/bin/perl GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/perl...Reading symbols from /usr/lib/debug/usr/bin/suidperl.debug...done. done. (gdb) r /tmp/1.pl /svn/testrepo Starting program: /usr/bin/perl /tmp/1.pl /svn/testrepo [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x7138af10 in ?? () (gdb) bt #0 0x7138af10 in ?? () #1 0x003bba01899e in run_cleanups (pool=0x8d7828) at memory/unix/apr_pools.c:2314 #2 apr_pool_destroy (pool=0x8d7828) at memory/unix/apr_pools.c:782 #3 0x003bba018975 in apr_pool_destroy (pool=0x7e1498) at memory/unix/apr_pools.c:779 #4 0x003bba018b84 in apr_pool_terminate () at memory/unix/apr_pools.c:605 #5 0x71e5153d in _wrap_apr_terminate (my_perl=, cv=) at core.c:2590 #6 0x003ae0aa6815 in Perl_pp_entersub (my_perl=0x603010) at pp_hot.c:2888 #7 0x003ae0aa4b06 in Perl_runops_standard (my_perl=0x603010) at run.c:40 #8 0x003ae0a4c5df in Perl_call_sv (my_perl=0x603010, sv=0x6c9190, flags=6) at perl.c:2721 #9 0x003ae0a4cb1d in Perl_call_list (my_perl=0x603010, oldscope=1, paramList=0x6c9388) at perl.c:5276 #10 0x003ae0a4e947 in perl_destruct (my_perl=0x603010) at perl.c:630 #11 0x00400e01 in main (argc=3, argv=0x7fffe5a8, env=0x7fffe5c8) at perlmain.c:126 [[[ 1.pl ]]] [aneyman@localhost ~]$ cat /tmp/1.pl #!/usr/bin/perl use SVN::Core; use SVN::Repos; die "Usage: 1.pl REPO TXN" unless $#ARGV == 0; my $repo = SVN::Repos::open($ARGV[0]) or die "Cannot open repo " . $ARGV[0]; exit 0; ]]] The same script works fine with the stock CentOS bindings. Regards, Alexey.
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 11:53:37 am Philip Martin wrote: > Alexey Neyman writes: > > 0x7138af10 in ?? () > > (gdb) bt > > #0 0x7138af10 in ?? () > > #1 0x003bba01899e in run_cleanups (pool=0x8d7828) at > > memory/unix/apr_pools.c:2314 > > #2 apr_pool_destroy (pool=0x8d7828) at memory/unix/apr_pools.c:782 > > #3 0x003bba018975 in apr_pool_destroy (pool=0x7e1498) at > > memory/unix/apr_pools.c:779 > > #4 0x003bba018b84 in apr_pool_terminate () at > > memory/unix/apr_pools.c:605 #5 0x71e5153d in > > _wrap_apr_terminate (my_perl=, cv= > out>) at core.c:2590 > > #6 0x003ae0aa6815 in Perl_pp_entersub (my_perl=0x603010) at > > pp_hot.c:2888 #7 0x003ae0aa4b06 in Perl_runops_standard > > (my_perl=0x603010) at run.c:40 #8 0x003ae0a4c5df in Perl_call_sv > > (my_perl=0x603010, sv=0x6c9190, flags=6) at perl.c:2721 > > #9 0x003ae0a4cb1d in Perl_call_list (my_perl=0x603010, oldscope=1, > > paramList=0x6c9388) at perl.c:5276 > > #10 0x003ae0a4e947 in perl_destruct (my_perl=0x603010) at perl.c:630 > > #11 0x00400e01 in main (argc=3, argv=0x7fffe5a8, > > env=0x7fffe5c8) at perlmain.c:126 > > The stacktrace may be more useful if you install the debuginfo packages > for subversion, apr and apr-util. They are installed, as you might see in the frames #1..#4. Apparently, it jumped somewhere outside of the function boundaries - so the frame #0 cannot determine the function name.
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 11:57:14 am Alexey Neyman wrote: > On Friday, September 12, 2014 11:53:37 am Philip Martin wrote: > > Alexey Neyman writes: > > > 0x7138af10 in ?? () > > > (gdb) bt > > > #0 0x7138af10 in ?? () > > > #1 0x003bba01899e in run_cleanups (pool=0x8d7828) at > > > memory/unix/apr_pools.c:2314 > > > #2 apr_pool_destroy (pool=0x8d7828) at memory/unix/apr_pools.c:782 > > > #3 0x003bba018975 in apr_pool_destroy (pool=0x7e1498) at > > > memory/unix/apr_pools.c:779 > > > #4 0x003bba018b84 in apr_pool_terminate () at > > > memory/unix/apr_pools.c:605 #5 0x71e5153d in > > > _wrap_apr_terminate (my_perl=, cv= > > out>) at core.c:2590 > > > #6 0x003ae0aa6815 in Perl_pp_entersub (my_perl=0x603010) at > > > pp_hot.c:2888 #7 0x003ae0aa4b06 in Perl_runops_standard > > > (my_perl=0x603010) at run.c:40 #8 0x003ae0a4c5df in Perl_call_sv > > > (my_perl=0x603010, sv=0x6c9190, flags=6) at perl.c:2721 > > > #9 0x003ae0a4cb1d in Perl_call_list (my_perl=0x603010, oldscope=1, > > > paramList=0x6c9388) at perl.c:5276 > > > #10 0x003ae0a4e947 in perl_destruct (my_perl=0x603010) at > > > perl.c:630 #11 0x00400e01 in main (argc=3, > > > argv=0x7fffe5a8, > > > env=0x7fffe5c8) at perlmain.c:126 > > > > The stacktrace may be more useful if you install the debuginfo packages > > for subversion, apr and apr-util. > > They are installed, as you might see in the frames #1..#4. Apparently, it > jumped somewhere outside of the function boundaries - so the frame #0 > cannot determine the function name. Actually, the program counter at the time of the crash does not belong to any shared library: (gdb) bt #0 0x7fffefefff10 in ?? () #1 0x003bba01899e in run_cleanups (pool=0x8d7828) at memory/unix/apr_pools.c:2314 #2 apr_pool_destroy (pool=0x8d7828) at memory/unix/apr_pools.c:782 #3 0x003bba018975 in apr_pool_destroy (pool=0x7e1498) at memory/unix/apr_pools.c:779 #4 0x003bba018b84 in apr_pool_terminate () at memory/unix/apr_pools.c:605 #5 0x71e5153d in _wrap_apr_terminate (my_perl=, cv=) at core.c:2590 #6 0x003ae0aa6815 in Perl_pp_entersub (my_perl=0x603010) at pp_hot.c:2888 #7 0x003ae0aa4b06 in Perl_runops_standard (my_perl=0x603010) at run.c:40 #8 0x003ae0a4c5df in Perl_call_sv (my_perl=0x603010, sv=0x6c9190, flags=6) at perl.c:2721 #9 0x003ae0a4cb1d in Perl_call_list (my_perl=0x603010, oldscope=1, paramList=0x6c9388) at perl.c:5276 #10 0x003ae0a4e947 in perl_destruct (my_perl=0x603010) at perl.c:630 #11 0x00400e01 in main (argc=3, argv=0x7fffe5a8, env=0x7fffe5c8) at perlmain.c:126 (gdb) info sharedlibrary FromTo Syms Read Shared Object Library 0x003acd600b00 0x003acd6198eb Yes /lib64/ld-linux-x86-64.so.2 0x003ae0a34d10 0x003ae0b31688 Yes /usr/lib64/perl5/CORE/libperl.so 0x003acfa03930 0x003acfa12938 Yes /lib64/libresolv.so.2 0x003ade604070 0x003ade6119f8 Yes /lib64/libnsl.so.1 0x003acda00de0 0x003acda01998 Yes /lib64/libdl.so.2 0x003ace603e70 0x003ace643fb8 Yes /lib64/libm.so.6 0x003add600c00 0x003add6059a8 Yes /lib64/libcrypt.so.1 0x003adf200e10 0x003adf201688 Yes /lib64/libutil.so.1 0x003ace205760 0x003ace2110c8 Yes /lib64/libpthread.so.0 0x003acde1ea60 0x003acdf4024c Yes /lib64/libc.so.6 0x003add203c40 0x003add2539e8 Yes /lib64/libfreebl3.so 0x71e49f10 0x71ef9c68 Yes /usr/local/lib64/perl5/auto/SVN/_Core/_Core.so 0x71bd1fd0 0x71c18f58 Yes /usr/lib64/libsvn_client-1.so.0 0x719b3af0 0x719be218 Yes /usr/lib64/libsvn_delta-1.so.0 0x717a9970 0x717ad198 Yes /usr/lib64/libsvn_fs-1.so.0 0x7159ce40 0x715a36e8 Yes /usr/lib64/libsvn_ra-1.so.0 0x7136e640 0x7138a5c8 Yes /usr/lib64/libsvn_repos-1.so.0 0x710cb930 0x7113f688 Yes /usr/lib64/libsvn_wc-1.so.0 0x70ea5c40 0x70eb2a48 Yes /usr/lib64/libsvn_diff-1.so.0 0x70bbc3e0 0x70c65be8 Yes /usr/lib64/libsvn_subr-1.so.0 0x70994ce0 0x7099a4d8 Yes /usr/lib64/libsvn_swig_perl-1.so.0 0x003acea02120 0x003acea0d3a8 Yes /lib64/libz.so.1 0x003bb9c091e0 0x003bb9c1c668 Yes /usr/lib64/libaprutil-1.so.0 0x003ad1603cd0 0x003ad161cc88 Yes /lib64/libexpat.so.1 0x003adc2286a0 0x003adc33e9d8 Yes /usr/lib64/libdb-4.7.so 0x003bba00b8
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 12:06:27 pm Alexey Neyman wrote: > On Friday, September 12, 2014 11:57:14 am Alexey Neyman wrote: > > On Friday, September 12, 2014 11:53:37 am Philip Martin wrote: > > > Alexey Neyman writes: > > > > 0x7138af10 in ?? () > > > > (gdb) bt > > > > #0 0x7138af10 in ?? () > > > > #1 0x003bba01899e in run_cleanups (pool=0x8d7828) at > > > > memory/unix/apr_pools.c:2314 > > > > #2 apr_pool_destroy (pool=0x8d7828) at memory/unix/apr_pools.c:782 > > > > #3 0x003bba018975 in apr_pool_destroy (pool=0x7e1498) at > > > > memory/unix/apr_pools.c:779 > > > > #4 0x003bba018b84 in apr_pool_terminate () at > > > > memory/unix/apr_pools.c:605 #5 0x71e5153d in > > > > _wrap_apr_terminate (my_perl=, cv= > > > optimized out>) at core.c:2590 > > > > #6 0x003ae0aa6815 in Perl_pp_entersub (my_perl=0x603010) at > > > > pp_hot.c:2888 #7 0x003ae0aa4b06 in Perl_runops_standard > > > > (my_perl=0x603010) at run.c:40 #8 0x003ae0a4c5df in Perl_call_sv > > > > (my_perl=0x603010, sv=0x6c9190, flags=6) at perl.c:2721 > > > > #9 0x003ae0a4cb1d in Perl_call_list (my_perl=0x603010, > > > > oldscope=1, paramList=0x6c9388) at perl.c:5276 > > > > #10 0x003ae0a4e947 in perl_destruct (my_perl=0x603010) at > > > > perl.c:630 #11 0x00400e01 in main (argc=3, > > > > argv=0x7fffe5a8, > > > > env=0x7fffe5c8) at perlmain.c:126 > > > > > > The stacktrace may be more useful if you install the debuginfo packages > > > for subversion, apr and apr-util. > > > > They are installed, as you might see in the frames #1..#4. Apparently, it > > jumped somewhere outside of the function boundaries - so the frame #0 > > cannot determine the function name. > > Actually, the program counter at the time of the crash does not belong to > any shared library I did a bit of bisection, and it turns out that 1.8.0 was working properly, but 1.8.1 and higher all segfault as described above. Hope that helps.
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 01:07:53 pm Ben Reser wrote: > On 9/12/14 12:16 PM, Alexey Neyman wrote: > > I did a bit of bisection, and it turns out that 1.8.0 was working > > properly, but 1.8.1 and higher all segfault as described above. > > > > Hope that helps. > > With the WANdisco builds or any builds? The only differences between 1.8.0 > and 1.8.1 that impacts the Perl bindings in any way was shutting up a > bunch of compiler warnings about the _ symbol (which Perl defines in their > header files and SVN also uses internally for different purposes) and a > bunch of documentation changes. Neither of which should have any > functional difference. > > If you're seeing this with the WANdisco builds I think this is something > wrong with those builds, which has been my inclination in looking into > this. I cannot try stock CentOS, as they don't have 1.8.x available. I did try building and installing 1.8.10 from source on the same machine, and it works fine. So, it doesn't look like a version-specific issue - rather, like a build environment issue for WANdisco subversion-perl RPMs. Is it possible that the build environment changed between 1.8.0 and 1.8.1? Regards, Alexey.
Re: Problem with subversion-perl RPMs from WanDisco?
On Friday, September 12, 2014 02:30:08 pm Alexey Neyman wrote: > On Friday, September 12, 2014 01:07:53 pm Ben Reser wrote: > > On 9/12/14 12:16 PM, Alexey Neyman wrote: > > > I did a bit of bisection, and it turns out that 1.8.0 was working > > > properly, but 1.8.1 and higher all segfault as described above. > > > > > > Hope that helps. > > > > With the WANdisco builds or any builds? The only differences between > > 1.8.0 and 1.8.1 that impacts the Perl bindings in any way was shutting > > up a bunch of compiler warnings about the _ symbol (which Perl defines > > in their header files and SVN also uses internally for different > > purposes) and a bunch of documentation changes. Neither of which should > > have any functional difference. > > > > If you're seeing this with the WANdisco builds I think this is something > > wrong with those builds, which has been my inclination in looking into > > this. > > I cannot try stock CentOS, as they don't have 1.8.x available. > > I did try building and installing 1.8.10 from source on the same machine, > and it works fine. So, it doesn't look like a version-specific issue - > rather, like a build environment issue for WANdisco subversion-perl RPMs. > > Is it possible that the build environment changed between 1.8.0 and 1.8.1? FWIW, I installed all the updates on this test machine and Perl bindings still crash. Does anyone else use Perl bindings from that WanDisco RPMs? Can anyone verify that it works using a simple testcase below on a CentOS 6.x? [[[ cat >/tmp/1.pl <<'EOF' #!/usr/bin/perl use SVN::Core; use SVN::Repos; die "Usage: 1.pl REPO TXN" unless $#ARGV == 0; my $repo = SVN::Repos::open($ARGV[0]) or die "Cannot open repo " . $ARGV[0]; exit 0; EOF chmod +x /tmp/1.pl svnadmin create /tmp/testrepo /tmp/1.pl /tmp/testrepo ]]] Regards, Alexey.
Re: Problem with subversion-perl RPMs from WanDisco?
On Wednesday, September 17, 2014 04:49:02 pm Nico Kadel-Garcia wrote: > >> I cannot try stock CentOS, as they don't have 1.8.x available. > >> > >> I did try building and installing 1.8.10 from source on the same > >> machine, and it works fine. So, it doesn't look like a version-specific > >> issue - rather, like a build environment issue for WANdisco > >> subversion-perl RPMs. > >> > >> Is it possible that the build environment changed between 1.8.0 and > >> 1.8.1? > > > > FWIW, I installed all the updates on this test machine and Perl bindings > > still crash. > > > > Does anyone else use Perl bindings from that WanDisco RPMs? Can anyone > > verify that it works using a simple testcase below on a CentOS 6.x? > > Can't speak to what they have there. My github hosted RPM building > tools at https://github.com/nkadel/subversion-1.8.x-srpm don't seem to > have this problem. That's nice to know, but I would like to avoid installing a build machine for creating binary RPMs from SRPM specs if possible. If you look at the quote above, I already mentioned that Subversion built from sources does not have this problem; the problem is with *binary* RPMs offered through opensource.wandisco.com. Regards, Alexey.
Re: Every Version of Every File in a Repository
On Wednesday, October 08, 2014 12:41:01 AM Branko Čibej wrote: > On 07.10.2014 22:36, Andreas Mohr wrote: > > Hi, > > > > That's certainly a somewhat tough one. > > > > > > I will get tarred and feathered here for my way of trying to solve this, > > and possibly even rightfully so, but... ;) > > Well, I certainly won't skin you alive for suggesting this; but ... I > would imagine that "git svn fetch" has to essentially do just what the > OP doesn't want to do, i.e., successively retreive each revision of > every file in the Subversion repository to populate the Git repository. > There's not much chance this would be faster than just doing the same > with Subversion, especially since, once you're done you /still/ have to > scan the files resulting Git repo. > > > Going back to the original question ... > > >Aside from the brute-force method of checking out the entire repository > >starting at revision 1 , performing a scan, updating to the next > >revision, > >and repeating until I reach the head, I don’t know of a way to do this. > > This is, in fact, likely to be (almost) the most efficient way to do > this, since you can just use the existing Subversion client to deal with > the repository contents and version discrepancies. > > But there is an alternative that might be more efficient in your case: > Create a dumpstream of the repository using "svnadmin dump", > non-incremental and not using deltas, then pipe the stream to a custom > tool that extracts the file contents the stream and either writes them > to disk, or passes them to your scanning tool in some other way. > > The reason why this could be faster than the checkout+repeated update is > that you do not have the overhead of a working copy, directory tracking, > property handling, etc. etc., and you can probably save on disk space by > keeping the file contents around only as long as they're being scanned. > It does mean that your custom tool will have to parse the dumpfile > format, but that's really not so hard, the format is quite simple, and > there are a number of example scripts that do that in our repository. > Another alternative is to use our API directly, possibly through one of > the bindings, to get file contents straight from the repository; but I > suspect it's harder than parsing the dump file. The Python bindings for parsing the dumpstream currently do not work as I described on svn-dev@ some time ago: the layer which does "thunking" of the C calls back to Python code is not implemented right now. As far as I can see, Perl/Ruby bindings have the same problem. That, and the way to create a stream in Python does not seem to be working - see the email I just sent to svn-dev@ a few minutes ago. Ironically, I found that when I tried to test the implementation of this "thunking" code for parsing the dumpstream :) Not sure if this affects Perl/Ruby. So, back to your advice - it's either using C library directly, or implementing the parser for the stream. Which isn't hard, I admit. Regards, Alexey. > > -- Brane
svn: E200030: sqlite: Expected database row missing
Hi, Was using svn 1.8.10, CentOS 6.5, Subversion RPM installed from opensource.wandisco.com. During 'svn switch', the following error pops up: svn: E200030: sqlite: Expected database row missing What I was doing: - checked out a project from /trunk - did some local changes - copied WC to a branch - attempted to switch to the branch If I repeat the 'svn switch ^/branches/mybranch' command, the first time switch proceeded a bit further - but later yields the same error. Subsequent runs yield this error immediately. I cannot reproduce this by repeating the 'checkout-modify-copy-switch' sequence, but I am keeping this working copy in case some information can be gleaned from it. Any ideas what may be wrong? Regards, Alexey.
Re: svn: E200030: sqlite: Expected database row missing
Any ideas on what may have caused this error, and what info can be obtained from the WC? On 10/09/2014 03:48 PM, Alexey Neyman wrote: Was using svn 1.8.10, CentOS 6.5, Subversion RPM installed from opensource.wandisco.com. During 'svn switch', the following error pops up: svn: E200030: sqlite: Expected database row missing What I was doing: - checked out a project from /trunk - did some local changes - copied WC to a branch - attempted to switch to the branch If I repeat the 'svn switch ^/branches/mybranch' command, the first time switch proceeded a bit further - but later yields the same error. Subsequent runs yield this error immediately. I cannot reproduce this by repeating the 'checkout-modify-copy-switch' sequence, but I am keeping this working copy in case some information can be gleaned from it. Any ideas what may be wrong? Regards, Alexey.
Re: svn: E200030: sqlite: Expected database row missing
On 10/13/2014 04:08 PM, Andreas Stieger wrote: Hello, On 09/10/14 23:48, Alexey Neyman wrote: What I was doing: - checked out a project from /trunk - did some local changes - copied WC to a branch - attempted to switch to the branch What exactly does "copied WC to a branch" mean in terms of svn or file system operations? It means exactly that, copying from a WC to a new directory in the repository: $ svn cp . ^/engr/aneyman/mybranch Regards, Alexey.
Re: svn: E200030: sqlite: Expected database row missing
On 10/14/2014 05:03 AM, Philip Martin wrote: Alexey Neyman writes: On 10/13/2014 04:08 PM, Andreas Stieger wrote: Hello, On 09/10/14 23:48, Alexey Neyman wrote: What I was doing: - checked out a project from /trunk - did some local changes - copied WC to a branch - attempted to switch to the branch What exactly does "copied WC to a branch" mean in terms of svn or file system operations? It means exactly that, copying from a WC to a new directory in the repository: $ svn cp . ^/engr/aneyman/mybranch That's a commit. Did it succeed? If so then the commit should contain a lot of information about the state of the working copy. What does the commit look like? What does 'svn log -v' show for the commit? Adds, modifications, copies, deletes, property changes, etc.? Yes, it is a commit and it has succeeded. If I check out a separate working copy from that new path, it is fine and contains all the modifications from the working copy thus checked in. Actually, looking at the commit I came up with a reproduction scenario: [[[ #!/bin/bash cd /tmp svnadmin create repo svn co file://`pwd`/repo wc svn mkdir wc/trunk wc/trunk/d1 wc/trunk/d2 echo hi > wc/trunk/d1/f1 svn add wc/trunk/d1/f1 svn ci -mm wc rm -rf wc svn co file://`pwd`/repo/trunk wc svn mv wc/d1/f1 wc/d2 svn rm wc/d1 svn cp -mm wc file://`pwd`/repo/newbranch svn switch file://`pwd`/repo/newbranch wc ]]] Running this script under SVN 1.8.10 gives the same "missing row" error. So, the problem operation is "copy a file from a directory, then delete that directory". Checked the issue using SVN trunk. It does not abort like 1.8.10, but it does report tree conflicts for d1/f1 and d1. I would say such conflicts should be resolved automatically, given that the working copy contains exactly the same changes as in the incoming edit. Figuring that out may not be trivial, though, as the copied directory may be arbitrarily deep. Regards, Alexey.
Re: svn: E200030: sqlite: Expected database row missing
On 10/15/2014 01:23 AM, Philip Martin wrote: Checked the issue using SVN trunk. It does not abort like 1.8.10, but it does report tree conflicts for d1/f1 and d1. I would say such conflicts should be resolved automatically, given that the working copy contains exactly the same changes as in the incoming edit. Figuring that out may not be trivial, though, as the copied directory may be arbitrarily deep. The difficulty here is that Subversion does not know that the repository tree was created as a commit from this working copy. Resolving the tree-conflicts automatically probably needs more complete move-tracking. I think this kind of conflicts happens with plain deletions as well. I don't have a trunk Subversion available right now to check though. I think theoretically Subversion could work its way up: first it could note that d1/f1 is removed in both local tree and the repository. Since it is the same change, it could then resolve that conflict. Then, seeing no other changes in d1 in the local tree - it could conclude that the deletion of the d1 directory is again the same operation - and resolve that conflict, too. Lather, rinse, repeat for any higher-level directories. Regards, Alexey.
Re: svn merge --reintegrate like diff
On 09/27/2016 01:46 AM, Daniel Shahaf wrote: Johan Corveleyn wrote on Mon, Sep 26, 2016 at 13:04:04 +0200: Maybe I'm missing something, but I don't understand why 'svn diff --old=TRUNK --new=BRANCH' would show you things that you previously merged from TRUNK to BRANCH. It should show exactly the content-wise difference between TRUNK and BRANCH, so if some content was merged from TRUNK to BRANCH, both should be identical on that point, and it shouldn't show up in 'diff'. That command would also show changes made on trunk that have not yet been merged to the branch. (E.g., if you ran it in on subversion's trunk and 1.9.x branch, it would show -SVN_VER_MINOR 10\n +SVN_VER_MINOR 9\n.) The OP asked for the changes merge would do, which is approximately --old=TRUNK@REV --new=BRANCH where REV is the youngest revision of trunk merged to the branch. ("Approximately" because this is inaccurate when cherry-picks or subtree merges hapepned.) There's one more issue with these approaches. ReviewBoard can be smart about displaying the moved/copied files. However: - If one does 'svn merge --reintegrate', Subversion will copy new files from the branch unchanged, and 'svn diff' will not show them (and hence, RB won't either) - or, with --show-copies-as-adds, it will show them as being added anew. For the review purposes, it would be better if instead of copying the file from the branch unchanged, Subversion would perform the same move on the trunk and apply the textual changes. - If you do 'svn diff --old=... --new=...', I believe the copy-from information is lost from the diff completely - and ReviewBoard will show all the moved files as adds/deletes, not showing diffs either. For now, I am using the attached script to perform this task. The workflow is: 1. Perform a merge to trunk. 2. Run the script (which attempts to find the original location in trunk for every copied file and replay the move on trunk). 3. 'rbt post'. The script is not perfect; it only operates at file level - so if there are renamed directories, the files inside them end up in 'R +' status (replaced with history) and ReviewBoard shows a spurious deletion for this file, in addition to a move/copy with changes. Regards, Alexey. #!/usr/bin/python3 # vim: set et sw=4 : import os import re import subprocess import sys allowed_paths = [ "/vendor/" ] debug = False def get_svninfo_value(svnlog, lookfor): for l in svnlog.splitlines(): if l.startswith(lookfor): return l[len(lookfor):] def get_common_part(f1, f2): l1 = f1.split('/') l2 = f2.split('/') for i in range(0, min(len(l1), len(l2))): if l1[i] != l2[i]: break; else: i = min(len(l1), len(l2)) + 1 return '/'.join(l1[0:i]) def get_original_path(f): copied_path = f rest = "" wcroot = None while True: if copied_path == wcroot: return f svnlog = subprocess.check_output(["svn", "info", copied_path], universal_newlines=True) if wcroot is None: wcroot = get_svninfo_value(svnlog, "Working Copy Root Path: ") # Path to WC rel_path = get_svninfo_value(svnlog, "Relative URL: ^") # Relative URL root_url = get_svninfo_value(svnlog, "Repository Root: ") # Repository root URL copy_url = get_svninfo_value(svnlog, "Copied From URL: ") # Copy-from URL if (rel_path is None or root_url is None): raise ValueError if (copy_url is not None): if (not(copy_url.startswith(root_url + '/'))): print("Invalid copy URL") raise ValueError break last_slash = copied_path.rindex("/") rest = copied_path[last_slash:] + rest copied_path = copied_path[:last_slash] rel_path += rest copy_url += rest svnlog = subprocess.check_output(["svn", "info", wcroot], universal_newlines=True) rel_root_path = get_svninfo_value(svnlog, "Relative URL: ^") # Relative URL for root of WC if rel_root_path is None: print("No root path found") raise ValueError lookfor = copy_url[len(root_url):] if debug: print('wcroot %s' % wcroot) print("root rel path {%s}" % rel_root_path) print("look for {%s}" % lookfor) try: svnlog = subprocess.check_output(["svn", "log", "-qv", f], universal_newlines=True) except subprocess.CalledProcessError: return f # Ok, even though inside a copied path, this path does not seem to be copied logrevs = svnlog.split("\n")[1:-1] while True: if lookfor.startswith(rel_root_path): orig = wcroot + lookfor[len(rel_root_path):] if debug: print("found local copy source for `%s': `%s' (lookfor `%s')" % (f, orig, lookfor)) return orig elif get_common_part(lookfor, rel_root_path): if debug:
Re: svn merge --reintegrate like diff
On 09/28/2016 09:49 AM, Stefan Sperling wrote: On Tue, Sep 27, 2016 at 10:12:28AM -0700, Alexey Neyman wrote: On 09/27/2016 01:46 AM, Daniel Shahaf wrote: Johan Corveleyn wrote on Mon, Sep 26, 2016 at 13:04:04 +0200: Maybe I'm missing something, but I don't understand why 'svn diff --old=TRUNK --new=BRANCH' would show you things that you previously merged from TRUNK to BRANCH. It should show exactly the content-wise difference between TRUNK and BRANCH, so if some content was merged from TRUNK to BRANCH, both should be identical on that point, and it shouldn't show up in 'diff'. That command would also show changes made on trunk that have not yet been merged to the branch. (E.g., if you ran it in on subversion's trunk and 1.9.x branch, it would show -SVN_VER_MINOR 10\n +SVN_VER_MINOR 9\n.) The OP asked for the changes merge would do, which is approximately --old=TRUNK@REV --new=BRANCH where REV is the youngest revision of trunk merged to the branch. ("Approximately" because this is inaccurate when cherry-picks or subtree merges hapepned.) There's one more issue with these approaches. ReviewBoard can be smart about displaying the moved/copied files. However: - If one does 'svn merge --reintegrate', Subversion will copy new files from the branch unchanged, and 'svn diff' will not show them (and hence, RB won't either) - or, with --show-copies-as-adds, it will show them as being added anew. For the review purposes, it would be better if instead of copying the file from the branch unchanged, Subversion would perform the same move on the trunk and apply the textual changes. - If you do 'svn diff --old=... --new=...', I believe the copy-from information is lost from the diff completely - and ReviewBoard will show all the moved files as adds/deletes, not showing diffs either. For now, I am using the attached script to perform this task. The workflow is: 1. Perform a merge to trunk. 2. Run the script (which attempts to find the original location in trunk for every copied file and replay the move on trunk). 3. 'rbt post'. The script is not perfect; it only operates at file level - so if there are renamed directories, the files inside them end up in 'R +' status (replaced with history) and ReviewBoard shows a spurious deletion for this file, in addition to a move/copy with changes. Regards, Alexey. Hi Alexey, Could you compile an SVN client from trunk and try some merges with it, and let me know how the merging of moves with the new conflict resolver (which is still work-in-progress) is working out for you? My goal is to make scripts like yours unnecessary. The current implementation does not yet detect moves which happened inside copies, but I hope to get that fixed before release. Thanks, Stefan I gave it a try (r1763039) and it is not different from what I see with 1.9.x: the files that were renamed on the branch are still copied from the branch, not renamed on the trunk. I.e., svn cp $SVNREPO/trunk $SVNREPO/branch/x svn co $SVNREPO/branch/x cd x svn mv foo.c bar.c vi bar.c svn ci cd .. rm -rf x svn co $SVNREPO/trunk cd trunk svn merge ^/branch/x svn info bar.c The last command shows bar.c as being copied, without any changes, from ^/branch/x/bar.c - rather than being copied from ^/trunk/bar.c and modified. And, since there are no changes in the diff, ReviewBoard shows nothing in the diff for bar.c. Regards, Alexey.
Re: svn merge --reintegrate like diff
On 10/01/2016 10:59 PM, Stefan Sperling wrote: On Sat, Oct 01, 2016 at 10:19:34PM -0700, Alexey Neyman wrote: On 09/28/2016 09:49 AM, Stefan Sperling wrote: Hi Alexey, Could you compile an SVN client from trunk and try some merges with it, and let me know how the merging of moves with the new conflict resolver (which is still work-in-progress) is working out for you? My goal is to make scripts like yours unnecessary. The current implementation does not yet detect moves which happened inside copies, but I hope to get that fixed before release. Thanks, Stefan I gave it a try (r1763039) and it is not different from what I see with 1.9.x: the files that were renamed on the branch are still copied from the branch, not renamed on the trunk. I.e., svn cp $SVNREPO/trunk $SVNREPO/branch/x svn co $SVNREPO/branch/x cd x svn mv foo.c bar.c vi bar.c svn ci cd .. rm -rf x svn co $SVNREPO/trunk cd trunk svn merge ^/branch/x svn info bar.c The last command shows bar.c as being copied, without any changes, from ^/branch/x/bar.c - rather than being copied from ^/trunk/bar.c and modified. And, since there are no changes in the diff, ReviewBoard shows nothing in the diff for bar.c. You'll have to produce some kind of tree conflict involving the renamed file. The run 'svn resolve' (or use the conflict prompt 'svn merge' opens for you). To be clear, 1.10.x will *not* change 'svn merge'. It changes post-merge behaviour during tree conflict resolution only. I'm afraid this won't fix your problem with ReviewBoard. Well, tree conflicts during merge is a separate pain point with SVN, but here I was referring specifically to the reviewability of the merges - so 1.10 will not make this script obsolete, unfortunately. Perhaps, the change in merge behavior can be made conditional on some command line option, e.g. 'svn merge --replay-merged-renames'? Also, I was just made aware that the script I posted has a shortcoming if the branch being merged has been previously merged into trunk, and then had seen more development on the branch (i.e., it is a branch for ongoing development, not "merge-and-delete" branch). Thing is, it is not very straightforward with Subversion to discover *where this file has been copied to*. For example, "I have a file foo.c in revision 111; under which name(s) is this file appearing in HEAD?" I think adding an ability to obtain this kind of information has been discussed in the past - is it on the table for 1.10? Regards, Alexey.
svn generates long lists of empty diffs with --diff-cmd
Hi, I noticed that SVN generates a lot of spurious diffs when using a --diff-cmd option that look like this: Index: arch/alpha/boot/tools/objstrip.c === Index: arch/alpha/boot/Makefile === Is it possible to suppress such empty diffs? This makes the tools like ReviewBoard (www.reviewboard.org) examine a lot of unchanged files and slows them down considerably and unnecessary. Regards, Alexey.
Re: svn generates long lists of empty diffs with --diff-cmd
On 04/20/2017 06:52 AM, Bert Huijben wrote: -Original Message- From: Alexey Neyman [mailto:sti...@att.net] Sent: donderdag 20 april 2017 02:59 To: users@subversion.apache.org Subject: svn generates long lists of empty diffs with --diff-cmd Hi, I noticed that SVN generates a lot of spurious diffs when using a --diff-cmd option that look like this: Index: arch/alpha/boot/tools/objstrip.c == = Index: arch/alpha/boot/Makefile == = Is it possible to suppress such empty diffs? This makes the tools like ReviewBoard (www.reviewboard.org) examine a lot of unchanged files and slows them down considerably and unnecessary. What argument do you pass to --diff-cmd ? svn diff --diff-cmd=diff Is it possible that the files are actually different, but your diff command reports that they are not different (e.g. after normalizing whitespace). I think this header is printed on changed files, right before invoking the diff command. No, plain 'svn diff' does not show any differences, not even properties. For example: [aneyman@rhel-build2 src]$ svn diff $URL/SETUP.bash SETUP.bash [aneyman@rhel-build2 src]$ svn diff --diff-cmd=diff $URL/SETUP.bash SETUP.bash Index: SETUP.bash === Alexey.
Re: svn generates long lists of empty diffs with --diff-cmd
On 04/20/2017 09:58 PM, Alexey Neyman wrote: On 04/20/2017 06:52 AM, Bert Huijben wrote: -Original Message- From: Alexey Neyman [mailto:sti...@att.net] Sent: donderdag 20 april 2017 02:59 To: users@subversion.apache.org Subject: svn generates long lists of empty diffs with --diff-cmd Hi, I noticed that SVN generates a lot of spurious diffs when using a --diff-cmd option that look like this: Index: arch/alpha/boot/tools/objstrip.c == = Index: arch/alpha/boot/Makefile == = Is it possible to suppress such empty diffs? This makes the tools like ReviewBoard (www.reviewboard.org) examine a lot of unchanged files and slows them down considerably and unnecessary. What argument do you pass to --diff-cmd ? svn diff --diff-cmd=diff Is it possible that the files are actually different, but your diff command reports that they are not different (e.g. after normalizing whitespace). I think this header is printed on changed files, right before invoking the diff command. No, plain 'svn diff' does not show any differences, not even properties. For example: [aneyman@rhel-build2 src]$ svn diff $URL/SETUP.bash SETUP.bash [aneyman@rhel-build2 src]$ svn diff --diff-cmd=diff $URL/SETUP.bash SETUP.bash Index: SETUP.bash === Some more information: - While the properties on the file itself are the same, the inherited properties differ (at least, svn:mergeinfo on the parent directory). - This behavior is observed on 1.10.0-dev, 1.9.x and 1.8.x. Subversion 1.7.x does not display these spurious "Index:" lines. Regards, Alexey.
Re: svn generates long lists of empty diffs with --diff-cmd
On 04/20/2017 10:44 PM, Alexey Neyman wrote: On 04/20/2017 09:58 PM, Alexey Neyman wrote: On 04/20/2017 06:52 AM, Bert Huijben wrote: -Original Message- From: Alexey Neyman [mailto:sti...@att.net] Sent: donderdag 20 april 2017 02:59 To: users@subversion.apache.org Subject: svn generates long lists of empty diffs with --diff-cmd Hi, I noticed that SVN generates a lot of spurious diffs when using a --diff-cmd option that look like this: Index: arch/alpha/boot/tools/objstrip.c == = Index: arch/alpha/boot/Makefile == = Is it possible to suppress such empty diffs? This makes the tools like ReviewBoard (www.reviewboard.org) examine a lot of unchanged files and slows them down considerably and unnecessary. What argument do you pass to --diff-cmd ? svn diff --diff-cmd=diff Is it possible that the files are actually different, but your diff command reports that they are not different (e.g. after normalizing whitespace). I think this header is printed on changed files, right before invoking the diff command. No, plain 'svn diff' does not show any differences, not even properties. For example: [aneyman@rhel-build2 src]$ svn diff $URL/SETUP.bash SETUP.bash [aneyman@rhel-build2 src]$ svn diff --diff-cmd=diff $URL/SETUP.bash SETUP.bash Index: SETUP.bash === Some more information: - While the properties on the file itself are the same, the inherited properties differ (at least, svn:mergeinfo on the parent directory). - This behavior is observed on 1.10.0-dev, 1.9.x and 1.8.x. Subversion 1.7.x does not display these spurious "Index:" lines. So, given the information above, can it be considered a bug/regression? Regards, Alexey. Regards, Alexey.
Empty Index lines
Hi, Is it possible to get rid of the lots of "Index:" headers in the output of the 'svn diff' where there is no actual difference between the files? I think this started with SVN 1.9 - 1.7 definitely did not do that, and neither did 1.8, as far as I remember. For example, I copied Linux kernel from one path to another and now 'svn diff' shows the Index: line on each copied path, e.g. $ svn di arch/x86 | head -20 Index: arch/x86/.gitignore === Index: arch/x86/Kbuild === Index: arch/x86/Kconfig === Index: arch/x86/Kconfig.cpu === Index: arch/x86/Kconfig.debug === Index: arch/x86/Makefile === Index: arch/x86/Makefile.um === Index: arch/x86/Makefile_32.cpu === Index: arch/x86/boot/.gitignore === Index: arch/x86/boot/Makefile === As you might imagine, this bloats the patch considerably and effectively makes it unreviwable. There is no changes in all these files, except that there is a parent directory that were copied. $ svn info . Path: . Working Copy Root Path: /home/aneyman/work/WC URL: $REPO/$PATH/linux-generic Relative URL: ^/$PATH/linux-generic Repository Root: $REPO Repository UUID: 12b7edc4-22db-4a73-a48b-88f3acc8629e Revision: 14230 Node Kind: directory Schedule: add Copied From URL: $REPO/$ORIGINAL_PATH Copied From Rev: 14230 Moved From: $ORIGINAL_PATH_IN_WC Last Changed Author: aneyman Last Changed Rev: 14227 Last Changed Date: 2017-12-05 16:40:55 -0800 (Tue, 05 Dec 2017) How to make SVN not emit this garbage? Regards, Alexey.
Show textual diff in a moved/copied file - how?
Hi all, I am trying to dig for some changes in a file that was moved a few times and 'svn diff' shows full "remove old location and add new location as if it were a new file" diffs, which are not helpful. Is there a way to make the diff show the changes as compared against the origin of the copy? I tried --notice-ancestry, does not help. I have a vague recollection that 'svn diff' used to show the changes in such copied files before - but I tried the small reproduction script below and it shows the same, both with 1.7.22/1.8.17/1.9.7/trunk: ---8<--- #!/bin/bash rm -rf /tmp/foo-{svn,wc} svnadmin create /tmp/foo-svn svn co file:///tmp/foo-svn foo-wc cd foo-wc echo foo > foobar svn add foobar svn ci -m "1" svn mv foobar barfoo echo bar >> barfoo svn ci -m "2" svn up svn diff -c 2 svn --version ---8<--- Diff output: ---8<--- Index: foobar === --- foobar (revision 1) +++ foobar (nonexistent) @@ -1 +0,0 @@ -foo Index: barfoo === --- barfoo (nonexistent) +++ barfoo (revision 2) @@ -0,0 +1,2 @@ +foo +bar ---8< Regards, Alexey.
Re: Show textual diff in a moved/copied file - how?
On 02/26/2018 12:18 AM, Stefan Sperling wrote: On Sun, Feb 25, 2018 at 11:38:03PM -0800, Alexey Neyman wrote: Hi all, I am trying to dig for some changes in a file that was moved a few times and 'svn diff' shows full "remove old location and add new location as if it were a new file" diffs, which are not helpful. Is there a way to make the diff show the changes as compared against the origin of the copy? I tried --notice-ancestry, does not help. Diff output changes depending on whether you pass a path to the file itself or to a parent of the file. Try: svn diff -c 2 barfoo I found this in the diff_renamed_file() test in diff_tests.py, see there for more examples. https://svn.apache.org/repos/asf/subversion/trunk/subversion/tests/cmdline/diff_tests.py You don't expect the end-user to read the test cases in the product to get these subtleties, do you? :) And, I find it quite counter-intuitive. I would expect --notice-ancestry at least to take ancestral relationship between these files into account; the currently shown diff is the same as if 'barfoo' were not copied but was created from scratch. Since you know all paths and revisions involved, you could also run: svn diff ^/foobar@1 ^/barfoo@2 Well, either of these approaches is not very convenient when there is a dozen moves & modifications in a single revision. Besides, the former (just passing the path) does not seem to work in all cases. In the real repository, I have two revisions that did the same thing: moved a directory and modified some files in the moved directory. The trick with passing the path to the file works for one of them, but not for the other - and I am at a loss why SVN treats these two differently. Here's where diff does not display the proper diff even when supplied with the path to the file: # The relevant fragment of a revision $ svn log -c 36 -v file://`pwd`/XX-svn A /trunk/XX/src/bin/more (from /vendor/:29) M /trunk/XX/src/bin/more/more.c # Passing the path to the directory that was copied: does not work $ svn di -c 36 file://`pwd`/XX-svn/trunk/XX/src/bin/more | grep -A 4 'Index: more.c' Index: more.c === --- more.c (nonexistent) +++ more.c (revision 36) @@ -0,0 +1,1894 @@ # Passing the path to the specific file: does not work $ svn di -c 36 file://`pwd`/XX-svn/trunk/XX/src/bin/more/more.c | grep -A 4 'Index: more.c' Index: more.c === --- more.c (nonexistent) +++ more.c (revision 36) @@ -0,0 +1,1894 @@ # Manual, file-by-file: works, but doesn't scale to revisions with lots of modifications $ svn di file://`pwd`/los178-svn{/vendor//more.c@29,/trunk/X/src/bin/more/more.c@36} | grep -A 4 'Index: more.c' Index: more.c === --- more.c (.../vendor/BSD/more/4.3Tahoe/more.c) (revision 29) +++ more.c (.../trunk/los178/src/bin/more/more.c) (revision 36) @@ -1,3 +1,11 @@ Regards, Alexey. I have a vague recollection that 'svn diff' used to show the changes in such copied files before - but I tried the small reproduction script below and it shows the same, both with 1.7.22/1.8.17/1.9.7/trunk: ---8<--- #!/bin/bash rm -rf /tmp/foo-{svn,wc} svnadmin create /tmp/foo-svn svn co file:///tmp/foo-svn foo-wc cd foo-wc echo foo > foobar svn add foobar svn ci -m "1" svn mv foobar barfoo echo bar >> barfoo svn ci -m "2" svn up svn diff -c 2 svn --version ---8<--- Diff output: ---8<--- Index: foobar === --- foobar (revision 1) +++ foobar (nonexistent) @@ -1 +0,0 @@ -foo Index: barfoo === --- barfoo (nonexistent) +++ barfoo (revision 2) @@ -0,0 +1,2 @@ +foo +bar ---8< Regards, Alexey.
Re: Show textual diff in a moved/copied file - how?
On 02/26/2018 01:49 AM, Stefan Sperling wrote: And, I find it quite counter-intuitive. I would expect --notice-ancestry at least to take ancestral relationship between these files into account; (I don't have time to look at the code right now, so I'm speculating a bit.) You're diffing *directories*, not files. There are separate client-side handlers for directory and file diffs which might not always have the same information available. E.g. it may not be feasible to trace the back the copy history of every child when diffing two directories. I am not that familiar to say why 'svn diff' behaves in the way it does, but it does look like it's contradicting the description in 'svn help diff': --notice-ancestry : diff unrelated nodes as delete and add Since 'svn diff' does not take the opposite option, '--ignore-ancestry', I'd say one would assume that 'svn diff' should diff *related* nodes textually, not *as delete and add*. Tracing each child may take some additional time, right, but between "fast and wrong" and "slow and correct" behaviors, I'd choose the latter :) Since you know all paths and revisions involved, you could also run: svn diff ^/foobar@1 ^/barfoo@2 Well, either of these approaches is not very convenient when there is a dozen moves & modifications in a single revision. Agreed. At least the file diffs allows you to 'zoom in', but it would be much better if there was a way to get the diff you want to see with just one command. If backwards compatibility of 'svn diff' behavior, or the performance impact of tracing every child, is a concern - is it possible to have 'svn diff' do such history tracing if enabled by some new option? Although, I cannot come up with a better name than 'svn diff --properly-diff-related-nodes'. Besides, the former (just passing the path) does not seem to work in all cases. In the real repository, I have two revisions that did the same thing: moved a directory and modified some files in the moved directory. The trick with passing the path to the file works for one of them, but not for the other - and I am at a loss why SVN treats these two differently. Here's where diff does not display the proper diff even when supplied with the path to the file: [... snip ...] I can't explain this one. It might be worth filing an issue about this problem in case you can come up with a standalone recipe to reproduce it. I found what triggers this behavior. This happens when the source of the copy is not the revision immediately preceding the revision being diffed. Here's the script for reproduction: ---8<--- #!/bin/bash r=`pwd`/foo-svn url=file://$r wc=`pwd`/foo-wc rm -rf $r $wc svnadmin create $r svn co $url $wc cd $wc echo "Initial content" > foo svn add foo svn ci -m "Initial import" # Source revision to be used in copy later srev=`svnlook youngest $r` if [ "$INSERT_EXTRA_REVISION" = "yes" ]; then svn mkdir somedir svn ci -m "Extra revision" fi svn cp $url/foo@$srev bar echo "Added line" >> bar svn ci -m "Copy + modify" cmrev=`svnlook youngest $r` svn diff -c $cmrev $url/bar@$cmrev ---8<--- And here is the output from the script: ---8<--- $ ./test.sh ... Index: foo === --- foo (.../foo) (revision 1) +++ foo (.../bar) (revision 2) @@ -1 +1,2 @@ Initial content +Added line $ INSERT_EXTRA_REVISION=yes ./test.sh ... Index: bar === --- bar (nonexistent) +++ bar (revision 3) @@ -0,0 +1,2 @@ +Initial content +Added line ---8<--- Why is the behavior different in these cases? Isn't that counter-intuitive as well that the diff's output depends on the source revision of the copy? Regards, Alexey.
Re: Show textual diff in a moved/copied file - how?
On 02/27/2018 03:26 AM, Johan Corveleyn wrote: On Tue, Feb 27, 2018 at 9:13 AM, Stefan Sperling wrote: On Mon, Feb 26, 2018 at 04:52:41PM -0800, Alexey Neyman wrote: Why is the behavior different in these cases? Isn't that counter-intuitive as well that the diff's output depends on the source revision of the copy? I think these differences in behaviour boil down to side-effects of the implementation. As I posted before in this thread, this problem was already noted and discussed before on dev@ (feel free to follow the links I posted :-)). But I'm happy this issue is brought back to the foreground, because I too consider this an issue and inconsistent behaviour from the user's perspective (regardless of the underlying implementation problem). Back in 2013, you, Stefan, wrote: On Tue, Jun 25, 2013 at 10:56 PM, Stefan Sperling wrote: On Tue, Jun 25, 2013 at 10:26:08PM +0200, Ben Reser wrote: Back to your issue. Since Subversion can't represent the copy as part of the diff it tries to do the interoperable thing which is to represent the addition of a new file (from a copy) as an addition. That's not quite true in general. It's more like: 1) If the copyfrom source is part of the operative revision range of the diff command, show a modification against the copyfrom source. Unless --show-copies-as-adds was passed, because then we always show copied files as an addition. 2) If the copyfrom source is not part of the operative revision range, history of the file isn't traced back to that revision, so it appears as an addition. It could be argued that 2) is weird special case, and that it should behave like 1) (i.e. trace back to the copyfrom source anyway) and only show an addition with --show-copies-as-adds. Johan pointed out that svnlook diff seems to traverse to the copyfrom source even in case 2). If this is indeed the case, these commands are now behaving in contradictory ways :( However, I think it's too late to change either command now. Thanks for bringing up this explanation. So the second inconsistency is because '-c X' actually defines operative range X-1:X and the source of the copy is X-2 in this case. Indeed, a lot of subtleties and inconsistencies that appear to be bugs. Is there ever going to be SVN 2.0 that can finally break these bug-for-bug compatibility promises? Is there a list of things that are going to be changed in 2.0? Regards, Alexey.
Weird behavior of `svn --non-interactive`
Hi all, I am encountering some weird behavior after upgrading my workstation to Ubuntu 18.10 - which also upgraded the SVN to version 1.10.0 (r1827917). An attempt to query anything from the server using the `--non-interactive` flag fails, unless there has been a recent "interactive session" with this server. aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/ svn: E170013: Unable to connect to a repository at URL 'svn://svn.lynx.com/lynxsecure' svn: E170001: Can't get username or password aneyman@yehat:~/work/lsk-ranges$ svn pl ^/ Properties on 'svn://svn.lynx.com/lynxsecure': reviewboard:url aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/ Properties on 'svn://svn.lynx.com/lynxsecure': reviewboard:url This happens during various actions by `rbt` (RBTools) which runs svn with --non-interactive flag. Note that the "interactive" run of svn does not even query the password - it happily uses the stored password and proceeds. Why isn't the non-interactive invocation doing the same? I also tried the development version of Subversion a couple of weeks ago; it has the same behavior. Regards, Alexey.
Re: Weird behavior of `svn --non-interactive`
On 2/26/19 12:07 AM, Stefan Sperling wrote: On Mon, Feb 25, 2019 at 05:41:13PM -0800, Alexey Neyman wrote: Hi all, I am encountering some weird behavior after upgrading my workstation to Ubuntu 18.10 - which also upgraded the SVN to version 1.10.0 (r1827917). An attempt to query anything from the server using the `--non-interactive` flag fails, unless there has been a recent "interactive session" with this server. aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/ svn: E170013: Unable to connect to a repository at URL 'svn://svn.lynx.com/lynxsecure' svn: E170001: Can't get username or password aneyman@yehat:~/work/lsk-ranges$ svn pl ^/ Properties on 'svn://svn.lynx.com/lynxsecure': reviewboard:url aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/ Properties on 'svn://svn.lynx.com/lynxsecure': reviewboard:url This happens during various actions by `rbt` (RBTools) which runs svn with --non-interactive flag. Note that the "interactive" run of svn does not even query the password - it happily uses the stored password and proceeds. Why isn't the non-interactive invocation doing the same? I also tried the development version of Subversion a couple of weeks ago; it has the same behavior. Regards, Alexey. I agree this looks like a bug. To find the bug we'll likely need to know which password store is actually being used by your installation of Subversion. plaintext? gpg-agent? gnome-keyring? kwallet? SVN configuration doesn't have the password store option specified, so I assume it is using the default - according to the comment in the .subversion/config, it is "gpg-agent,gnome-keyring,kwallet". I have kwallet installed and configured with empty master password. I also have gpg-agent and gnome-keyring installed, but I don't think I ever used either of them on that machine. How can I check which store was SVN actually trying to use at the time it happens? Regards, Alexey.
Re: Weird behavior of `svn --non-interactive`
On 2/26/19 8:42 PM, Alexey Neyman wrote: On 2/26/19 12:07 AM, Stefan Sperling wrote: On Mon, Feb 25, 2019 at 05:41:13PM -0800, Alexey Neyman wrote: Hi all, I am encountering some weird behavior after upgrading my workstation to Ubuntu 18.10 - which also upgraded the SVN to version 1.10.0 (r1827917). An attempt to query anything from the server using the `--non-interactive` flag fails, unless there has been a recent "interactive session" with this server. aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/ svn: E170013: Unable to connect to a repository at URL 'svn://svn.lynx.com/lynxsecure' svn: E170001: Can't get username or password aneyman@yehat:~/work/lsk-ranges$ svn pl ^/ Properties on 'svn://svn.lynx.com/lynxsecure': reviewboard:url aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/ Properties on 'svn://svn.lynx.com/lynxsecure': reviewboard:url This happens during various actions by `rbt` (RBTools) which runs svn with --non-interactive flag. Note that the "interactive" run of svn does not even query the password - it happily uses the stored password and proceeds. Why isn't the non-interactive invocation doing the same? I also tried the development version of Subversion a couple of weeks ago; it has the same behavior. Regards, Alexey. I agree this looks like a bug. To find the bug we'll likely need to know which password store is actually being used by your installation of Subversion. plaintext? gpg-agent? gnome-keyring? kwallet? SVN configuration doesn't have the password store option specified, so I assume it is using the default - according to the comment in the .subversion/config, it is "gpg-agent,gnome-keyring,kwallet". I have kwallet installed and configured with empty master password. I also have gpg-agent and gnome-keyring installed, but I don't think I ever used either of them on that machine. How can I check which store was SVN actually trying to use at the time it happens? Actually, it is gpg-agent. I went to $HOME/.subversion/auth/svn.simple; somehow there is a mixture of files using gpg-agent and gnome-keyring authentication methods. I found the one corresponding to the repository URL; it has: K 8 passtype V 9 gpg-agent Funny thing is, I found another file in that directory that refers to the same repository, just with a different URL (one is using the FQDN of the Subversion server, the other just the hostname). That other file uses gnome-keyring and it seems to work fine: aneyman@yehat:~/work/lsk-pristine$ svn --non-interactive pl svn://svn/lynxsecure Properties on 'svn://svn/lynxsecure': reviewboard:url aneyman@yehat:~/work/lsk-pristine$ svn --non-interactive pl svn://svn.lynx.com/lynxsecure svn: E170013: Unable to connect to a repository at URL 'svn://svn.lynx.com/lynxsecure' svn: E170001: Can't get username or password How does SVN decide when to use gpg-agent and when to use gnome-keyring? By the way, I am running KDE so I'd assume the kwallet would be the default - but it isn't... If you need me to build Subversion with some kind of debugging patch, let me know. Regards, Alexey.
Re: Weird behavior of `svn --non-interactive`
On 2/26/19 11:22 PM, Stefan Sperling wrote: Since you have several password stores running, the best bet to get reliable behaviour is to pick the one you actually want to use and configure it in the ~/.subversion/config file. By the way, 'svn auth' will list cached credentials and also show which password cache they are stored in. Ok, I left only gpg-agent configured since its the one giving the trouble. Issue still reproducible after a period of SVN inactivity. Regards, Alexey.
"svnlook pl --revprop" does not work on transactions?
Hi all, It seems that I encountered a bug in 'svnlook pl --revprop': it fails with the following message: $ svnlook pl --revprop -t 10547-86b /svn/test-svn svnlook: Invalid revision number '-1' Observed with Subversion 1.6.6. Looks like offending code is this block in do_plist(): == if (path != NULL) ... else { SVN_ERR(svn_fs_revision_proplist(&props, c->fs, c->rev_id, pool)); revprop = TRUE; } == Note that it always uses svn_fs_revision_proplist(), even when '-t TXN' is passed. In this case, c->rev_id == SVN_INVALID_REVNUM (-1), and svn_fs_revision_proplist() rightfully fails. Shouldn't it be using svn_fs_txn_proplist() instead, just as get_property() chooses between svn_fs_txn_prop() and svn_fs_revision_prop()? This conditional is still in /trunk, so most likely, bug is also reproducible with top-of-trunk Subversion. Regards, Alexey.
"svnlook pl --revprop" does not work on transactions?
Hi all, It seems that I encountered a bug in 'svnlook pl --revprop': it fails with the following message: $ svnlook pl --revprop -t 10547-86b /svn/test-svn svnlook: Invalid revision number '-1' Observed with Subversion 1.6.6. Looks like offending code is this block in do_plist(): == if (path != NULL) ... else { SVN_ERR(svn_fs_revision_proplist(&props, c->fs, c->rev_id, pool)); revprop = TRUE; } == Note that it always uses svn_fs_revision_proplist(), even when '-t TXN' is passed. In this case, c->rev_id == SVN_INVALID_REVNUM (-1), and svn_fs_revision_proplist() rightfully fails. Shouldn't it be using svn_fs_txn_proplist() instead, just as get_property() chooses between svn_fs_txn_prop() and svn_fs_revision_prop()? This conditional is still in /trunk, so most likely, bug is also reproducible with top-of-trunk Subversion. Regards, Alexey. signature.asc Description: This is a digitally signed message part.
Re: "svnlook pl --revprop" does not work on transactions?
Olivier, Actually, I tried to check the presence of some custom properties (say, foo:bar and foo:baz), so svnlook info does not print them. These properties are essentially boolean (they're either present or not). So, I tried to check them all in one run. Instead, I am doing several "svnlook pg --revprop -t TXN", which works. Regards, Alexey. On Saturday 27 February 2010 06:36:44 am Olivier Dehon wrote: > On Fri, 2010-02-26 at 22:15 -0800, Alexey Neyman wrote: > > Hi all, > > > > It seems that I encountered a bug in 'svnlook pl --revprop': it fails > > with the following message: > > > > $ svnlook pl --revprop -t 10547-86b /svn/test-svn > > svnlook: Invalid revision number '-1' > > Looks like you need to use "svnlook info -t" to access the values of the > properties that are going to be set on the revision-to-be (svn:log, > svn:author, ...) > > -Olivier >
Re: "svnlook pl --revprop" does not work on transactions?
On Saturday 27 February 2010 11:35:24 am Lieven Govaerts wrote: > > Note that it always uses svn_fs_revision_proplist(), even when '-t TXN' > > is passed. In this case, c->rev_id == SVN_INVALID_REVNUM (-1), and > > svn_fs_revision_proplist() rightfully fails. > > > > Shouldn't it be using svn_fs_txn_proplist() instead, just as > > get_property() chooses between svn_fs_txn_prop() and > > svn_fs_revision_prop()? > > Think so. Do you want to submit a patch to fix this issue? Sure, attached. Alexey. Index: subversion/svnlook/main.c === --- subversion/svnlook/main.c (revision 917057) +++ subversion/svnlook/main.c (working copy) @@ -1687,7 +1687,14 @@ } else { - SVN_ERR(svn_fs_revision_proplist(&props, c->fs, c->rev_id, pool)); + if (! c->is_revision) + { + SVN_ERR(svn_fs_txn_proplist(&props, c->txn, pool)); + } + else + { + SVN_ERR(svn_fs_revision_proplist(&props, c->fs, c->rev_id, pool)); + } revprop = TRUE; }
Re: "svn log" via svnserve is letting me see things it shouldn't, but "svn ls" works as I expect
On Wednesday 03 March 2010 01:14:38 pm Stefan Sperling wrote: > On Wed, Mar 03, 2010 at 03:01:22PM -0600, Reid Priedhorsky wrote: > > In particular, log messages to files not in > > /cyclingproject/public should not be available. > > Log message are not per file. They are per revision. > They aren't tied to any particular path. > Off-hand I cannot think of a way to prevent them from being seen. Using svnadmin dump/load sequence piped through "svndumpfilter include --drop-empty-revs /cyclingproject/public" instead of svnsync would probably help. Regards, Alexey. signature.asc Description: This is a digitally signed message part.
How to use txdelta_window?
Hi all, I am trying to check for some content changes in the pre-commit script, and for that I am trying to employ the apply_textdelta method in the editor. From the comments in the code (e.g. in python bindings), I understand that the "window handler" callback should be called repeatedly and the last call would pass None (NULL in C) as the window pointer. However that does not seem to be the case. I tried creating the following editor and invoking it from pre-commit scripts: def add_file(self, path, parent_baton, copyfrom_path, copyfrom_revision, file_pool): if copyfrom_path is not None: sys.stderr.write(("@@ add_file %s (from %s:%d)\n" % (path, copyfrom_path, copyfrom_rev))) else: sys.stderr.write(("@@ add_file %s\n" % (path))) return [ path ] def apply_textdelta(self, file_baton, base_checksum): sys.stderr.write(("@@ apply_textdelta to %s\n" % (file_baton[0]))) def winhnd(window, btn=file_baton, rcv=self): if window is not None: sys.stderr.write(("@@ delta_window on %s - not null\n" % (btn[0]))) else: sys.stderr.write(("@@ delta_window on %s - null\n" % (btn[0]))) return winhnd def open_file(self, path, parent_baton, base_revision, file_pool): sys.stderr.write(("@@ open_file %s\n" % (path))) return [ path ] def close_file(self, file_baton, text_checksum): sys.stderr.write(("@@ close_file %s\n" % (file_baton[0]))) return With that script, I get the output as below: @@ open_file foo/bar1 @@ apply_textdelta to foo/bar1 @@ delta_window on foo/bar1 - null @@ close_file foo/bar1 @@ open_file foo/bar2 @@ apply_textdelta to foo/bar2 @@ delta_window on foo/bar2 - null @@ close_file foo/bar2 That is, the "window handler" is always called with None object as the text delta. Am I doing something wrong? Could the change content be accessed from the pre-commit script? Regards, Alexey.
Re: Could not un- and re- link ~/.subversion/config
"File exists" error means that the file is re-created after you delete it but before 'ln'. The fact that you were able to achieve re-linking by combining two commands into one (rm ... && ln ...) suggests that there is something invoking 'svn' periodically (cron job?). As far as I know, Subversion does not have any special logic for config file protection. However, it does re-create this file with default settings if it does not find one when 'svn' command is invoked. Regards, Alexey. On Thursday 04 March 2010 05:13:49 am Alan Brogan wrote: > Hello the list, > > I just lost a few hours trying to do this: > > $ cd ~/.subversion > $ rm -f config > $ ln -s /path/to/another/config . > > The link command kept failing, because "File exists" >WTF ? > > Turns out I cannot remove ~/.subversion/* in one command, as some other > process is protecting them from deletion (but not from editing). > > Eventually I did figure out the workaround, which is simply to join > them into one command $ rm -f config && ln -s /path/to/another/config . > > But it was a £*({ing annoying few hours between problem arising and > finding solution (and even more annoying that the solution was so > simple) > > I can find no documentation of this feature at > http://svnbook.red-bean.com/en/1.5/ > http://subversion.apache.org/faq.html > http://mail-archives.apache.org/mod_mbox/subversion-users/ > so, I would like to ask: > > a) where is it documented ? > b) can it be extended to other user-configurable directories ? > > Thank you for reading this far, and any writing you might be able to > do.
Re: How to use txdelta_window?
Figured it out; I used svn_repos_replay, which does not send deltas. Everything works with svn_repos_replay2. Regards, Alexey. On Wednesday 03 March 2010 05:34:02 pm Alexey Neyman wrote: > Hi all, > > I am trying to check for some content changes in the pre-commit script, > and for that I am trying to employ the apply_textdelta method in the > editor. From the comments in the code (e.g. in python bindings), I > understand that the "window handler" callback should be called > repeatedly and the last call would pass None (NULL in C) as the window > pointer. > > However that does not seem to be the case. I tried creating the > following editor and invoking it from pre-commit scripts: > > def add_file(self, path, parent_baton, copyfrom_path, > copyfrom_revision, file_pool): if copyfrom_path is not None: > sys.stderr.write(("@@ add_file %s (from %s:%d)\n" % (path, > copyfrom_path, copyfrom_rev))) else: > sys.stderr.write(("@@ add_file %s\n" % (path))) > return [ path ] > > def apply_textdelta(self, file_baton, base_checksum): > sys.stderr.write(("@@ apply_textdelta to %s\n" % (file_baton[0]))) > def winhnd(window, btn=file_baton, rcv=self): > if window is not None: > sys.stderr.write(("@@ delta_window on %s - not null\n" % > (btn[0]))) else: > sys.stderr.write(("@@ delta_window on %s - null\n" % (btn[0]))) > return winhnd > > def open_file(self, path, parent_baton, base_revision, file_pool): > sys.stderr.write(("@@ open_file %s\n" % (path))) > return [ path ] > > def close_file(self, file_baton, text_checksum): > sys.stderr.write(("@@ close_file %s\n" % (file_baton[0]))) > return > > With that script, I get the output as below: > > @@ open_file foo/bar1 > @@ apply_textdelta to foo/bar1 > @@ delta_window on foo/bar1 - null > @@ close_file foo/bar1 > @@ open_file foo/bar2 > @@ apply_textdelta to foo/bar2 > @@ delta_window on foo/bar2 - null > @@ close_file foo/bar2 > > That is, the "window handler" is always called with None object as > the text delta. Am I doing something wrong? Could the change content > be accessed from the pre-commit script? > > Regards, > Alexey.