Re: SVN authentication is not working please help to implement this

[Daniel Shahaf]

Eichner, Siegfried wrote on Tue, 02 Jul 2019 07:30 +:
> Hi Team,
> 
> SVN authentication is not working please help to implement this. If a 
> checklist exist, i will do all check.
> The authentication was working before move to the new Server,
> The Apache is loading modules, but the author is not listet in history 
> and windows is not poping up by the connectionwindow to login. Thanks a 
> lot in advance for your helping.

That sounds like the new server allows both authenticated and anonymous
access.  Try disabling anonymous access.

Re: What does "Revision X doesn't match existing revision Y" mean?

[Daniel Shahaf]

Brett Cook wrote on Fri, 21 Jun 2019 15:45 +00:00:
> Now that I understand the issue, I don't believe there is any 
> workaround for the issue that will allow the builds to work with SVN 
> 1.7 (or above) the way they did with SVN version 1.6.  As such, I will 
> work with my team to try and make the changes necessary to work with 
> later SVN versions.

Thanks for sharing the diagnosis.  You should be able to use
svn:externals instead of the manual delete-and-checkout procedure.

Cheers,

Daniel

Re: What does "Revision X doesn't match existing revision Y" mean?

[Daniel Shahaf]

Brett Cook wrote on Fri, 21 Jun 2019 04:57 +00:00:
> I’m doing a checkout using ‘svn co URL DEST’ and get an error that says 
> “Revision X doesn't match existing revision Y in DEST”.

For future searchers' benefit, the error code should have been E155000.
(more below)

> I’ve tried figuring out what causes this and why an old version of SVN
> (svn, version 1.6.17 (r1128011) compiled Jun 2 2011, 23:35:08) works
> and a newer version (svn, version 1.11.1 (r1850623) compiled Jan 9
> 2019, 19:28:50 on x86-microsoft-windows) does not, but have not been
> successful.
> 
> When searching for the answer, I get a lot of information about this 
> happening during a merge, but have yet to find anything related to 
> checkout.
> 
> 
> I’m trying to find an explanation of a) what causes this and b) if 
> there are ways to have SVN not worry about it.

It's a client-side error.  I think it means working-copy recorded data
doesn't match data newly received over the wire, but I haven't had time
to confirm this.

The relevant client-side library was rewritten between 1.6 and 1.7,
which would explain why 1.6 and 1.11 behave differently.

Note that 1.11.x is no longer supported; only 1.9.x/1.10.x/1.12.x are
currently supported.

In 1.12.x (and probably 1.11.x too) the error should only be raised if
DEST existed and was a working copy before the 'svn co', so try specifying
a new or empty directory as DEST instead.

Cheers,

Daniel

Re: configure not finding serf lib

[Daniel Shahaf]

Michael Mueller wrote on Wed, 15 May 2019 20:34 +00:00:
> dev@cardamom:~/wrk/sandbox/svn/serf-1.3.9> scons 
> OPENSSL=/home/dev/wrk/openssl_build/save/1.0.2n APR=/usr/local/apr 
> APU=/usr/local/apr
> 
> 
> 
> Ran "scons install" as root.

Does serf's test suite pass?

> dev@cardamom:~/wrk/sandbox/svn/subversion-1.12.0> strings 
> /usr/local/lib/libserf-1.so.1.3.0 | grep serf_context_create
> 
> serf_context_create_ex
> 
> serf_context_create
> 
> 

It's a shot in the dark, but I would guess you have serf-1.2 or older in your 
linker's path, e.g., in /usr/lib/.

> From what I can tell from reading the configure script, a test file is 
> being compiled to see if serf_context_create is in the library it 
> compiles with. I have not been able to get the script to divulge which 
> lib it is using yet. __ __

Check config.log, in particular the command used to link the test
executable, and run it manually with verbose output enabled.

Re: E175012 and E175002 when trying to commit

[Daniel Shahaf]

Christoph Vogtländer wrote on Tue, 02 Apr 2019 09:02 +00:00:
> This is reproducible. Running with de_DE.UTF-8 (my default LANG), svn 
> will try to submit the change but runs into a time-out. Running with 
> en_GB.UTF-8, svn fails with error 413.
> 
> Why is subversion acting differently with different language setting on 
> the client side?
> 

I'm not sure, especially given that both locales are UTF-8 ones.  A few
shots in the dark:

- Try '-F /dev/null' instead of '-F svn-commit.tmp'.
- Is there any non-ASCII in the output of `svn st -q`?  (`svn st -q | sed 's/[ 
-~]//g' | grep .`; that's left bracket, space, minus, tilde, right bracket)
- Try LC_ALL=C and LC_ALL=C.UTF-8.

> Any ideas what can be done to successfully commit the changes?

Depends.  If you want to make the changes in one commit, you'll need to
increase the timeouts in server and client sufficiently (that includes
any proxies).  A workaround is to separate the changes to several
commits: use 'svn commit -- ./foo ./bar' instead of 'svn commit -- ./'.

I'm not sure why you'd get a 413 on the request to !svn/me, though.  (to
the list) Are any requests made to !svn/me other than the initial POST?

Re: svn log -r based on a start date suddenly no longer returns any revisions

[Daniel Shahaf]

Nate Kerkhofs wrote on Fri, 29 Mar 2019 09:08 +00:00:
> Checking the tag revisions done yesterday through the other unit tests, 
> all the dates of the revisions are monotonically increasing, with time 
> between dates ranging from 2 seconds to 2 hours, but always increasing 
> compared to the previous date. I'm not sure if any older revisions 
> might be causing the issue, but I'm assuming that any commits that have 
> been done a day or earlier before the issues occurred probably didn't 
> cause it, because I assume they'd have triggered the issue earlier. I'm 
> not sure about that last thing, but it seems logical.

I don't think that's right, actually.  Suppose that revisions 0:100 have
monotonically-increasing dates, then somebody creates a revision with an
old date (r101), then somebody creates a normal r102, then development
goes on at a rate of one revision per day. Because of how binary search
works, the backdated r101 would have no effect on the resolution of "in
the distant past" dates for a long time (and even once it does fire, it
will afterwards go latent again, because of how the math works).

You probably have a large chunk of backdated revisions, not just one.

Try running these commands:

svn info --show-item=revision -r "{2008-01-01T00:00:00}" ^/
svn info --show-item=revision -r "{2019-02-20T10:21:03}" ^/

I think you might find that the first of these two commands gives a
revision number more recent than you'd expect, probably for the reasons
explained in Andreas's first reply.

> In case it could actually be one of the earlier revisions, are there 
> any tricks or software I can use to test whether revisions are broken 
> in this manner?

Iterate the revisions from 0 to HEAD and check if their svn:date values
are monotonically increasing.  It's all of 10 lines of code in any
scripting language.  (If you use 'propget svn:date' you won't even need
a date arithmetic library, because sorting ISO-8601 data as ASCII
strings sorts it chronologically.)

Cheers,

Daniel

P.S. (Humour detour: rfc2550)

Re: --normalize-probs doesn't do its thing

[Daniel Shahaf]

Thorsten Goetzke wrote on Tue, 26 Mar 2019 10:40 +00:00:
> Hello,
> 
> I am evaluating a migration for some very old repositories to subversion 
> 1.10. These are currently running using subversion 1.7, but where most 
> likely created a much older version. I use the binaries provided by wandisco
> 
> svnadmin, version 1.10.3 (r1842928)
> 
> My current migration strategy is to apply a dump load cyrcle using the 
> current svnadmin tool
> Seems to work most repositories but for one particualar theres a problem.
> 
> I am using svnadmin load --normalize-props uls/ -F ../uls.txt
> 
> And i get:
> <<< Started new transaction, based on original revision 18083
>   * editing path : 
> uls/trunk/uls/TestAutomation/UMS_Testmanagement/02_Implementation/java/TestManagementUI
>  
> ...svnadmin: E125005: A property with invalid line ending found in 
> dumpstream; consider using --normalize-props while loading.

This error code is specifically caused by an svn:* property whose value
contains a CR byte (0x0D).  It could be either a revprop or a versioned
property.  In either case, --bypass-prop-validation should allow the
load to proceed (but the invalid data will remain in the history of the
new repository).

(We should make that error message show the name of the property, 

> Are there some catches on how to use --normalize-probs?

Note that it's spelled "props" (short for "properties").

Re: version control model of Subversion source code

[Daniel Shahaf]

I think wuzhouhui wasn't asking about branching models in general, but
specifically about what model the Subversion project uses for versioning
its own source code (https://svn.apache.org/repos/asf/subversion/).

wuzhouhui, your description is accurate.  In Subversion's tree
changes are committed first to trunk and then backported to release
branches via the STATUS files.  The process is described in HACKING at
.

Cheers,

Daniel

Paul Hammant wrote on Wed, 20 Mar 2019 15:21 +00:00:
> You can do any branching model you like with subversion. That said, I 
> maintain that Trunk-Based Development is the best model, and note that 
> there are merge issues with other branching models in rare 
> circumstances [ref 
> ].
> 
> On Wed, Mar 20, 2019 at 6:43 AM wuzhouhui  
> wrote:
> > Hi,
> > 
> >  This question is unrelated to Subversion's usage or issues, but
> >  related to version control model of Subversion source code. Let
> >  me explain in detail.
> > 
> >  As far as I know, new code is committed to trunk firstly, then
> >  cherry-picked to release-branch in necessary, so the branch-view
> >  of Subversion is following (cp stands for cherry-pick):
> > 
> >  ---oo--- 1.10.x
> >  / /
> >  cp cp
> >  / /
> >  o-oo-o-o trunk
> >  \ \ \
> >  cp cp cp
> >  \ \ \
> >  o-o---o 1.11.x
> > 
> >  And tags are always created on release-branch.
> > 
> >  Am I right?

Re: File capacity limitation in theory in SVN

[Daniel Shahaf]

Mark Phippard wrote on Wed, 20 Mar 2019 11:53 +00:00:
> * While there are no limits on number of files or total size,

Not to nitpick, but since there's no such thing as "unlimited capacity",
let me be one level more accurate:

Internally Subversion stores file size using signed 64-bit integers, so
the theoretical limit on the size of any one file or one directory
representation is 2**63-1 bytes.

The size of a directory representation is proportional to the sum of the
lengths of filenames within it.

Re: svn:E155007:None of the tarets are working copies

[Daniel Shahaf]

Fairchild, Gregory J - OASAM OCIO CTR wrote on Mon, 11 Mar 2019 22:22 +00:00:
> "You can avoid this by throwing a mutex around the svn(1) call (see 
> flock(1))."  Being that I'm pretty new to subversion, you lost me here.

It's possible for the post-commit process of rN to run in parallel to
the post-commit process of rN+1.  If that happens, it's possible for the
'svn update' processes launched by the two post-commit processes to
run in parallel.  If that happens, at least one of the two 'svn update'
processes will error out, probably with E155004 ("working copy is
already locked").

In short, post-commit hooks need to be written to account for the fact
that Subversion does not sequence or serialize hook invocations.  In
your case, that means changing «svn update $args» to «flock … svn update
$args» in order to ensure that concurrent post-commit hook runs will run
'svn update' in sequence, rather than in parallel.  (The runs might
_still_ be out of order — I mean, rN+1 before rN — but your hook is
indifferent to the $REV parameter.)

flock(1) is simply a program that takes a lock, aka a mutex.  It's not
standardized so you'll have to look up the flags on your system.

Cheers,

Daniel

P.S.  Having said all that, look up svnpubsub and svnwcsub in tools/ in
  the source tree.  They do basically what you are after with that hook.

> -Original Message-
> From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] 
> Sent: Saturday, March 09, 2019 7:22 AM
> To: Ryan Schmidt; Fairchild, Gregory J - OASAM OCIO CTR
> Cc: Subversion Users
> Subject: Re: svn:E155007:None of the tarets are working copies
...
> While I'm here, let me point out that post-commit runs for different
> revisions may happen in parallel or even out of order.  In this case, the
> worst that could happen is that the svn(1) invocation in the second run
> will error out with E155004 and the error will be marshalled back to the
> user who committed.  You can avoid this by throwing a mutex around the
> svn(1) call (see flock(1)).

Re: svn:E155007:None of the tarets are working copies

[Daniel Shahaf]

Ryan Schmidt wrote on Sat, 09 Mar 2019 08:43 +00:00:
> On Mar 8, 2019, at 14:24, Fairchild, Gregory J - OASAM OCIO CTR wrote:
> > Post-commit looks like this:
> > #!/bin/sh
> > REPOS="$1"
> > REV="$2"
> > /usr/bin/svn update /var/apache2/2.4/htdocs/ >> 
> > /rpool/svn/subversion/repository/logs/post-commit.log
> >  
> > Any help will be appreciated.  Thanks
> 
> The first question I would ask is: Does the directory 
> /rpool/svn/subversion/repository/logs exist? If not, create it.

That should solve the immediate problem, but you should plan to move
that directory to another place.

/rpool/svn/subversion/repository is a repository, so you shouldn't
create files or directories under it except those specifically described
in the documentation, for forward compatibility with future versions of
Subversion (which may start using $REPOS_DIR/logs/ in some way or
another).

While I'm here, let me point out that post-commit runs for different
revisions may happen in parallel or even out of order.  In this case, the
worst that could happen is that the svn(1) invocation in the second run
will error out with E155004 and the error will be marshalled back to the
user who committed.  You can avoid this by throwing a mutex around the
svn(1) call (see flock(1)).

Cheers,

Daniel

Re: Replacing directory by circular symlink produces malformed XML

[Daniel Shahaf]

Denis Excoffier wrote on Sat, 02 Mar 2019 09:17 +00:00:
> What would be the best strategy? I can figure out this one:
> 
> 1) Within the working copy, delete the existing tree (not svn delete)
> 2) Extract the new tree at the same place
> 3) Interpret the result of 'svn status' in terms of 'svn add' and
> 'svn delete' to apply on the working copy (details omitted here)
> 4) Commit
> 

That should work, though the 'svn status' will be expensive because the
usual optimization (not read()ing a file in full if its size+mtime
haven't changed) won't kick in.

In step 3 you can do 'svn add --force ./'.

There's an addremove branch [1] that attempts to implement this, but I
don't know what its state is.  (It's named after the analogous hg command.)

[1] https://svn.apache.org/viewvc/subversion/branches/addremove/

> The difficulty is to make sure that all peculiarities of a file system
> changes (say: only files, folders and symlinks) are handled properly.
> I don't talk about I/O errors of course.

Yeah, I see no reason why it shouldn't be possible to version circular
symlinks.  You can certainly create them with svnmucc, for example.

Cheers,

Daniel

Re: How to open specified inside .svn

[Daniel Shahaf]

wuzhouhui wrote on Sat, 02 Mar 2019 04:29 +00:00:
> In case of someone have interests, you can find my customized Subversion
> in https://github.com/wuzhouhui/subversion.

I think you want svn_io_check_resolved_path() there, no svn_io_check_path().

> Please forgive me about using Git to version control Subversion :-)

FYI, there is https://github.com/apache/subversion/.

Cheers,

Daniel

Re: Replacing directory by circular symlink produces malformed XML

[Daniel Shahaf]

Branko Čibej wrote on Fri, Mar 01, 2019 at 12:08:41 +0100:
> On 01.03.2019 11:53, Stefan Sperling wrote:
> > On Thu, Feb 28, 2019 at 10:44:25PM +0100, Denis Excoffier wrote:
> >> Hello,
> >>
> >> The situation of the replacement of a directory by a circular symlink
> >> generates an XML fragment which is invalid, hence triggers a failure
> >> within the calling system. See hereafter the recipe.
> >>
> >> Starting from a clean folder, do the following:
> >>
> >> % svn mkdir 1
> >> A 1
> >> % rmdir 1
> >> % ln -s 1 1
> >> % touch 2
> >> % svn status --xml
> >> 
> >> 
> >>  >>path=".">
> >>  >>path="1">
> >>  >>item="obstructed"
> >>revision="-1"
> >>props="none">
> >> 
> >> 
> >> svn: E62: Can't open directory '/Users/dexco/svntest/svn/1': Too many 
> >> levels of symbolic links
> >> %
> >>
> >> There is a missing end tag for  and .
> >> Another issue is missing output: path '2' is not listed.
> >>
> >> The correction of these issues will be much appreciated (i use 1.11.1).
> >>
> >> Regards,
> >>
> >> Denis Excoffier.
> > Hi Denis,
> >
> > This problem is not specific to symbolic links.
> > There are quite a few cases where --xml produces invalid XML
> > when it runs into some kind of error. Perhaps the command
> > line client should be fixed to close open XML tags when an
> > error occurs, though this also risks people or scripts not
> > noticing such errors.
> >
> > I suppose that, ideally, our XML output would embed errors
> > inside the XML stream in a well-defined manner, as well as
> > printing errors on stderr.
> >
> > So fixing this would require some non-trivial amount of effort.
> > Would you have time and skills to work on this issue?
> 
> My advice is to leave well enough alone. Any callers of our tools
> _should_ check the exit code before assuming that the output is valid.
> Just streaming the output through a SAX parser isn't the best strategy.

But there could have been tens of modified or unversioned s
before the circular symlink.  The size of output is O(N) where N is the
number of interesting items found in the wc walk (= the number of lines
in the non-XML `svn st` output).  Internally we always process trees
streamily; shouldn't we assume API consumers process trees streamily too?

Secondly, I don't understand why there's an error at all.  Shouldn't it
just report an obstruction (dirent's svn_kind_t value changed) and move on?

Cheers,

Daniel

Re: Re: How to open specified inside .svn

[Daniel Shahaf]

wuzhouhui wrote on Fri, Mar 01, 2019 at 17:46:55 +0800:
> > -Original Messages-
> > From: "Branko Čibej" 
> > Sent Time: 2019-03-01 17:16:40 (Friday)
> > To: users@subversion.apache.org
> > Cc: 
> > Subject: Re: How to open specified inside .svn
> > 
> > There are no such generic functions. The svn_wc API isn't really meant
> > to be public, so you'll have to write your own access functions. Look at
> > how the svn_client implementations do it, or for example
> > svn_wc__get_pristine_contents_by_checksum in svn_wc_private.h.
> 
> Actually, I'm hacking Subversion client command line tool, so it
> doesn't matter whether the API is public or private.

The difference isn't just visibility.  We don't promise compatibility
for private APIs, not even across patch releases in the same minor line
(1.A.x and 1.A.y).

Re: SVN keyword replacement

[Daniel Shahaf]

Mark Phippard wrote on Tue, 19 Feb 2019 20:53 +00:00:
> It ought to cover everything. If this looks like it is going to take a 
> considerable amount of your time then you might want to post what you 
> plan to do on the dev@ mailing list. There is no point spending time on 
> this if someone plans to veto it. As I noted, there are reasons some 
> might object to this idea. If it is a fairly quick patch then you might 
> as well let the patch show what you have in mind.

You probably want to use one of the svn_*_skip_ancestor() functions in
svn_dirent_uri.h.  You can look at 'svn info' for how to compute the wc
root path, if it isn't already available.

That said, I don't see anything preventing us from specifying a boolean
svn:x-branchroot nodeprop in 1.12.  We don't have to nail down the API,
or to add subcommands, or anything else; we can just specify it and see
what the ecosystem comes up with... though we _could_, say, introduce
an $X-branchrootrelativeurl$ keyword alongside it.

The X- in the names is to make them "experimental", i.e., covered by limited
compatibility promises.  With our recent shift to LTS and non-LTS releases,
we can start using non-LTS releases as vehicles for getting wider feedback
on ideas, by shipping those ideas as "experimental".

Cheers,

Daniel

Re: SVN keyword replacement

[Daniel Shahaf]

Mark Phippard wrote on Tue, 19 Feb 2019 18:30 +00:00:
> One idea that occurred to me would be adding a new value of %W that is 
> similar to %P except the path is relative to the root of the working 
> copy. Assuming that is generally the root of your project, such as 
> /trunk this would give the sort of path you desire. I do not know if 
> there would be any general objections to a value that varied so easily 
> based on something as arbitrary as the working copy root though. That 
> said, given that keywords are only expanded in the working copy it does 
> not seem unreasonable to hand this decision over to the user.

I'm not a fan of this.  The working copy root can be either above or
below the usual/customary level for creating the tag: when rolling a
Subversion release my wcroot may be ^/subversion,
^/subversion/branches/1.11.x, or
^/subversion/branches/1.11.x/tools/dist.  Ultimately, it's a random
environmental value, like ~/.subversion/config settings.  Any workflow
that diffs these files using anything other 'svn diff' — for example, diff(1)
or rsync — is going to run into the differences.  (For example, our release

There were discussions of this back in 2011/2012, for example, about
recursing upwards to the nearest path-wise ancestor that has an
svn:branchroot=yes property set, or the nearest path-wise ancestor that
is a copyfrom-wise descendant of trunk (i.e., has the same node-id as
trunk's noderev). [1]  This has the advantage that the keyword will be
expanded in the same way in all working copies, and even in
mod_dav_svn [2].

Cheers,

Daniel

https://svn.haxx.se/dev/archive-2012-07/0168.shtml
(message-id 4ce3f25b-e509-4743-b839-7d60c1797...@snakebite.org)

[2] https://subversion.apache.org/docs/release-notes/1.8#davkeywordexpansion
(we need a search engine for our release notes...)

Re: Homebrew SVN 1.11 not working

[Daniel Shahaf]

Stefan Sperling wrote on Wed, 16 Jan 2019 16:13 +0100:
> On Wed, Jan 16, 2019 at 09:39:12AM -0500, Mark Phippard wrote:
> > line 1760: none: command not found
> 
> Our configure script sets the variables which point to python, ruby,
> or perl to "none" by default. There is faulty logic somewhere in the
> swig bindings build which doesn't detect the non-presence of those binaries
> and still allows an attempt to compile bindings regardless. This should
> be fixed so that the make swig-* targets exit early in this situation.

Isn't that exactly what r1850519 does?

Re: Problems with using a symbolic link for .svn folder on TSVN

[Daniel Shahaf]

Oscar Lee wrote on Wed, 09 Jan 2019 19:10 +0100:
> My company uses TortoiseSVN internally to keep our files updated. The .svn
> folder for the project I have is massive (250GB) and as such I had to move
> it off to an external HDD. I created a symbolic link to the new location so
> that TortoiseSVN 'should' still continue to work.

How large are the working copy files not under the .svn/ directory?

If they're substantially smaller than 250GB, you might be running into this:
https://subversion.apache.org/docs/release-notes/1.7#wc-pristines

> I managed to run a clean-up, but when I tried to revert a file, it gave me
> an error 'Failed to run the WC DB work queue associated with (file)" and
> "Can't move (tmp file) to ... (original file): The system cannot move the
> file to a different disk drive".
> 
> I found that this error is caused by Windows not letting a file be renamed
> while it is being moved (
> https://docs.microsoft.com/en-us/previous-versions/ms837428(v=msdn.10)).
> Does anyone know a solution to this?

> Why is this an issue that only occurs with a symbolic link setup?

Subversion assumes that it is possible to atomically move a file from
the .svn directory to the working copy's checked out files.  That would
not possible when the .svn directory is on a different drive / filesystem.

Re: Bug report: An error occurred during authentication

[Daniel Shahaf]

Jan Marti wrote on Sun, 06 Jan 2019 00:32 +0100:
> But now, with Win10 instead of Win7 as reverse proxy, i get that error
> message:
> 
>  
> 
> "An error occurred during authentication"

Check the error logs of both the proxy server and the backend httpd
server for more detailed error messages.

Re: svn diff line numbers

[Daniel Shahaf]

abirami_n wrote on Wed, 02 Jan 2019 02:06 -0700:
> Hi,
> 
> I am using svn diff command to get the diff between two files from
> subversion.
> 
> I get the chunk header for the changed chunk.
> 
> But I am not finding any option to print line numbers.
> 
> Is it possible to get individual line numbers for all the changed lines?

The hunk (sic) header already includes that information: in "-a,b +c,d",
A and C are line numbers.  The output is in the standard unidiff format,
so if you want to add line numbers to every line, just pipe 'svn diff''s
output to any unidiff-aware program that shows or adds line numbers.
(I'm not aware of any off the top of my head, but it would be trivial to
write one.)

Cheers,

Daniel

Re: svnsync: E175002: REPORT request on '//!svn/rev/' failed

[Daniel Shahaf]

Nick Burkitt wrote on Fri, 28 Dec 2018 10:19 -0800:
> I contacted CollabNet, and while they didn't find a solution to the 
> svnsync problem per se, they did provide instructions for creating a 
> local backup using "snvrdump dump" and "svnadmin load."

Interesting.  'svnrdump dump' and 'svnsync sync' use exactly the same
wire API (svn_ra_replay_range()), so it's surprising that one of them
works and one doesn't.

> Thanks again for your help!

You're welcome.

Cheers,

Daniel

Re: Issue with binaries generated(subversion 1.10.2) after compilation on AiX 7.2

[Daniel Shahaf]

Prannoy K.v wrote on Fri, 28 Dec 2018 05:53 +:
>  
> Hi Daniel,
> Thank you for your help. I wanted to know if I am doing any mistake 
> during compilation or is it an Issue with AiX or Subversion ?

Did you see the comment in the source code I referred you to?  I'm
guessing the issue is simply a combination of two factors: (1) you
tested in a temporary mount point, (2) aix returns a different error
code than the one Subversion expects.

> If it is an issue with AiX and Subversion is it resolved in 1.10.3 or
> any other versions ?

I haven't committed the patch I posted, but if it's correct we can
commit it and backport it to the next release.  But then again, if my
guess is correct, the problem only affects temporary mount points to
begin with.

Cheers,

Daniel

Re: Issue with binaries generated(subversion 1.10.2) after compilation on AiX 7.2

[Daniel Shahaf]

Daniel Shahaf wrote on Thu, Dec 27, 2018 at 15:04:09 +:
> In subversion/libsvn_subr/io.c, in svn_io_file_flush_to_disk(), there's
> an APR_STATUS_IS_EINVAL() check.  Try making it check
> APR_STATUS_IS_EBADF() as well and handle it the same way (assuming "Bad
> file number" is strerror(3) of EBADF)

For clarity, I meant this:

[[[
Index: subversion/libsvn_subr/io.c
===
--- subversion/libsvn_subr/io.c (revision 1849781)
+++ subversion/libsvn_subr/io.c (working copy)
@@ -2505,6 +2505,10 @@ svn_error_t *svn_io_file_flush_to_disk(apr_file_t
  ignore the error. */
   if (rv == -1 && APR_STATUS_IS_EINVAL(apr_get_os_error()))
 return SVN_NO_ERROR;
+#ifdef _AIX
+  if (rv == -1 && APR_STATUS_IS_EBADF(apr_get_os_error()))
+return SVN_NO_ERROR;
+#endif
 
   if (rv == -1)
 return svn_error_wrap_apr(apr_get_os_error(),
]]]

> --- read the comment there while
> you're at it.  For context, creating new revisions relies on being able
> to flush the revision files to disk.

Re: Issue with binaries generated(subversion 1.10.2) after compilation on AiX 7.2

[Daniel Shahaf]

Prannoy K.v wrote on Thu, 27 Dec 2018 13:35 +:
> Hi Team, 
> I tried compiling subversion 1.10.2 on AiX 7.2 machine .

Note that 1.10.3 has been released.

> After "make" command generated binaries work well in the folder where it 
> is generated, whereas if we move binaries to a different location I get 
> the error :- 
> 
> svnadmin: E09: Can't write '/temp1110/home/Repo_test/db/current' 
> atomically 
> 
> svnadmin: E09: Can't flush file '/temp1110/home/Repo_test/db' to 
> disk: Bad file number  

In subversion/libsvn_subr/io.c, in svn_io_file_flush_to_disk(), there's
an APR_STATUS_IS_EINVAL() check.  Try making it check
APR_STATUS_IS_EBADF() as well and handle it the same way (assuming "Bad
file number" is strerror(3) of EBADF) --- read the comment there while
you're at it.  For context, creating new revisions relies on being able
to flush the revision files to disk.

Cheers,

Daniel

Re: svnsync: E175002: REPORT request on '//!svn/rev/' failed

[Daniel Shahaf]

Nick Burkitt wrote on Fri, 14 Dec 2018 12:36 -0800:
> Committed revision 69.
> Copied properties for revision 69.
> Transmitting file data ...
> Committed revision 70.
> Copied properties for revision 70.
> svnsync: E175002: REPORT request on '//!svn/rev/71' failed
> PS E:\svn>
>  end 
> 
> I'd be okay with losing the first 70 revisions, but I have no idea how to
> get past this point.

If you run 'svnsync sync file:///E:/svn/foo' again at this point, does
it fail again in the same way?

Re: svnsync: E175002: REPORT request on '//!svn/rev/' failed

[Daniel Shahaf]

Nick Burkitt wrote on Fri, 14 Dec 2018 12:36 -0800:
> The operation proceeds without problems until it gets to revision 71, at
> which point it fails with the error "svnsync: E175002: REPORT request on
> '//!svn/rev/71' failed". I've looked at all the related answers I
> could find, but none of them address this specific problem.
> 

The error code is SVN_ERR_RA_DAV_REQUEST_FAILED, which doesn't add
information.

> I'd be okay with losing the first 70 revisions, but I have no idea how to
> get past this point.

I wouldn't resort to that yet.  Do you have previous backups of these
revisions?  Does «svn checkout %SOURCE_REPO%@71» succeed?  If either of
these works, you should be able to work around the error in r71.
Furthermore, —

> The master is on a CollabNet CloudForge account, so I have no visibility
> into the server side - not even SVN version. I can view revs 69 through 72
> using the repo browser without detecting any problems.
> 

The standard next step is to look in the server's log for errors.  If
you don't have access to it yourself, then you should contact your
hosting provider.  If you do this, you hopefully wouldn't have to
discard the first 70 revisions (not to mention that based on the
information available, we can't rule out the possibility that _later_
revisions will have the same error).

> The client is TortoiseSVN (command line utilities):
> PS E:\svn> svnsync --version
> svnsync, version 1.11.0 (r1845130)
> compiled Oct 30 2018, 21:50:05 on x86-microsoft-windows
> 
> How do I go about analyzing this problem? Are there diagnostic tools? Log
> files? Guys who just know the answer?

As I said, my first recommendation is to talk to your hosting provider.
If you do this, you might skip the middleman and ask them to run
'svnadmin dump --deltas foo' and send you the resulting file.  If you
they don't figure it out, _then_ would be the time to start doing
fancier things (wireshark, workarounds using previous backups and/or
'svn checkout', etc).

Cheers,

Daniel

Re: Can SVN pull request same as GIT?

[Daniel Shahaf]

Nico Kadel-Garcia wrote on Sat, 01 Dec 2018 09:25 -0500:
> On Sat, Dec 1, 2018 at 4:00 AM Stefan Sperling  wrote:
> > Note that we use such a workflow in SVN itself: When we backport changes
> > to stable branches and a merge conflict occurs, we prepare a branch.
> > Voting happens in a file called STATUS, and once the change is approved,
> > a bot known as 'svn-role' will merge it. See the log here for example:
> > http://svn.apache.org/viewvc/subversion/branches/1.11.x/STATUS
> 
> But that is a branch, with its changes recorded in the single upstream
> repository. There is no split brain between repositories to resolve.

Yes, that's exactly the point.  The easiest way to implement a "pull
request" workflow in Subversion is to use multiple branches within
a single repository.

At the data model level, pull requests are feature branches.  The
differences are in the surrounding social patterns and in tooling built
around the "pull request - review - merge" workflow.  Such tooling could
be implemented on top of Subversion 1.1.0 (sic), and has been.

> The identity, contents, and order of every individual change is there
> already. Resolving that with the individual changes and change history
> of another repository is well, that's where I'd expect life to get
> both difficult and dangerous.

Using multiple repositories and porting commits between them amounts to
implementing a DVCS (such as git-svn) on top of Subversion.  However, as
several people already pointed out, there's no need to be distributed in
order to support pull requests.  One only needs to be able to pass
patches around; branches achieve that.

Cheers,

Daniel

Re: Subversion Web Management/Self-service Site

[Daniel Shahaf]

Shane Brodie wrote on Wed, 14 Nov 2018 13:18 -0600:
> My question is: Is there a web-site based management application which 
> allows the following features?
> 
> -  User password changes for Subversion access
> -  User access: read/write
> -  Management of users/groups
> -  Repository creation/removal

> Looking for something simple, up-to-date, hopefully supported directly 
> by Apache Subversion team.

The Subversion team does not release _any_ web-based application other
than the file browser built into mod_dav_svn (I mean, the code behind
https://svn.apache.org/repos/asf).  There are some third-party applications
whose teams intersect with the Subversion development team, of course.

Cheers,

Daniel

Re: Error: Could not access revision times

[Daniel Shahaf]

Nathan Hartman wrote on Fri, 09 Nov 2018 09:27 -0500:
> On Thu, Nov 8, 2018 at 4:24 PM Daniel Shahaf  wrote:
> 
> > It's a wild guess but perchance it's a 'svn co ${URL}@{DATE}' and the
> > binary search (to resolve {DATE} to a number) is failing due to authz?
> >
> 
> Or due to non-chronological commits? (Just another wild guess.)

Might be commits without svn:date.

Non-chronological commits shouldn't result in an error; just in
a possibly-wrong answer.  (There might not _be_ a correct answer,
of course.)

Re: Error: Could not access revision times

[Daniel Shahaf]

Mark Phippard wrote on Wed, 07 Nov 2018 08:20 -0500:
> Yes, the user that got this error was using SVNKit.  I do not know if it
> was via Jenkins, Eclipse or something else and that is why I am not sure
> what the command was.  I asked to recreate using SVN command line so we
> could better understand what triggered it.

To better understand what triggers the error we should patch 
subversion/mod_dav_svn/reports/dated-rev.c
to log the error that libsvn_repos returns there, and probably the
input date too.

It's a wild guess but perchance it's a 'svn co ${URL}@{DATE}' and the
binary search (to resolve {DATE} to a number) is failing due to authz?

Re: subversion 1.11.0 on FreeBSD 11.2: ... does not support the HTTP/DAV protocol

[Daniel Shahaf]

Dr. Rolf Jansen wrote on Tue, 06 Nov 2018 00:27 -0200:
> FYI, RFC 2518 has been obsoleted since 11 years by RFC 4918, and the 
> relevant chapter moved to 10.1: 
> https://tools.ietf.org/html/rfc4918#section-10.1, and yes there is still 
> written "the DAV server MUST return the DAV header". However, the RFC 
> does not require the DAV client to bail out if the server fails to do 
> so, this was your choice.

There are two schools of thought: one, "be strict in what you send and
lenient in what you accept"; two, "be strict in what you send and strict
in what you accept".  This list is not a good place to have
a discussion about which one is better in general.  Instead, let's focus on
the specific problem at hand.  If you have an idea for how we might continue
operating with GitHub without regressing the failure mode that that error
message was added to fix, we're all ears.

The obvious idea is to wait a few days for GitHub to make the one-line
change to their server code which will fix the issue.

Cheers,

Daniel

Re: Problems accessing GitHub's SVN-bridge with SVN 1.11

[Daniel Shahaf]

Thorsten Schöning wrote on Sun, 04 Nov 2018 18:57 +0100:
> Guten Tag Branko Čibej,
> am Sonntag, 4. November 2018 um 17:47 schrieben Sie:
> 
> > I'm not sure what you mean by "handles more than only DAV successfully"
> 
> I thought it might be possible that GitHub answers differently but
> properly, because the other check mentioned something about HTTP v2.

The terminology is a bit unfortunate here.

There's a HTTP/2 protocol, rfc7540, but it has *nothing* to do with what
Subversion design documents refer to as "HTTPv2".  The latter is simply
a different way for libsvn_ra_serf to communicate with mod_dav_svn, but
it was designed and implemented in terms of HTTP/1 requests and responses.
(Of course, one ought to be able to run HTTPv2 over HTTP/2 if one uses
serf and httpd versions that support the latter.)

In hindsight, we should have chosen a different name for that protocol revision.

Re: Problems accessing GitHub's SVN-bridge with SVN 1.11

[Daniel Shahaf]

Thorsten Schöning wrote on Sun, Nov 04, 2018 at 16:42:11 +0100:
> Guten Tag Thorsten Schöning,
> am Sonntag, 4. November 2018 um 16:05 schrieben Sie:
> 
> >> The server at '[...]' does not support the HTTP/DAV protocol.
> 
> Others have the same problem and while it is true that GitHub might
> have implemented something on their own, it might help to have a look
> at the changes between 1.10 and 1.11 regarding the protocol.

That took exactly two minutes to grep, diff, and blame.

See https://svn.apache.org/r1825302.  1.11 has new behaviour whereby it
deliberately errors out if the HTTP response does not include a "DAV:"
header, apparently in to improve the failure mode on URLs that are not
Subversion repository URLs.

> I've already written to the GitHub support but nobody seemed to care yet.

Well, we can't fix this issue on our end.  Our client code works with
our server code, but there is no way for us to ensure that our client
code (continues to) work with third-party server reimplementations.
That said, if any GitHub staff are reading this, you're welcome to
contact us on the dev@ list and we'll see what we can do.

Cheers,

Daniel
(We can't just "remove the new check" because that would regress the
failure mode that check was added to improve.)

Re: svndumpfilter and svnsync?

[Daniel Shahaf]

Chris wrote on Tue, 30 Oct 2018 13:36 +:
> One thing that was a bit annoying was when the dumpfilter threw an error 
> because of a source of a file was missing when I filtered out a certain 
> path and it turned out it had been copied to another location. The error 
> message only prints out the missing source and not the destination, so I 
> had to go into the repo to check the revision it crashed on to find the 
> copy destination and add it to my filter list. Would have been nice if 
> the error message could list both the source and the destination.

Fixed in https://svn.apache.org/r1845261.

Re: Repository became corrupt on commit

[Daniel Shahaf]

Alan Spark wrote on Wed, 24 Oct 2018 08:57 +0100:
> Hi Daniel,
> 
> I didn't get anywhere with the perl script. /usr/bin is definitely in
> the path and that is where svnlook is, and the SVNLOOK variable was
> already set... anyway I gave up and went for your alternative.
> 

Odd.  My first assumption is that the subshell on line 39 isn't behaving
as expected.  If you run `svnlook tree --full-paths --show-ids -r 719
/path/to/repository /trunk/scripts/script.py`, does print a path and a
node-rev id string (which looks like "a.b.c/d-e")?

> Firstly I must apologise as I stated revision 728 in my original email
> but it looks like this was on one of our experimental copies of the
> repository that we no longer have. I do still have a copy of the
> broken repository at revision 719. With that in mind, I ran this:
> 
> grep -a '^type:' /path/to/repository/db/revs/719
> type: file
> type: dir
> 
> I also ran on the last working revision:
> 
> grep -a '^type:' /path/to/repository/db/revs/718
> type: file
> type: dir
> type: dir
> type: dir
> type: dir
> type: dir
> 

These all look correct.  I assume that when you do a normal 'verify'
run, revisions 1:718 (inclusive) are verified successfully and 719
errors out, correct?  I.e., it prints "Verifying revision 718" and then
"Error verifying revision 719".

There's a slim chance that those "type:" lines had a NUL byte tacked on,
or something else that got lost in the translation to email.  You can
rule out this remote possibility by piping the grep to xxd(1) (just
'| xxd', no option flags needed).

Otherwise, could you please confirm that the error you're getting on
r719 is the same one you posted for r728 (except for the revision
number, of course)?

> I note that your path had a /0/ after /revs/ but this appears to be empty.
> 

It's normal for that path not to exist, particularly in older repositories.  It
shouldn't exist empty, but that's harmless.

> I think I have confirmed that our MPM is prefork:
> 
> a2query -M
> prefork
> 
> I hope this helps. Let me know if you need me to check anything else.

Yes, it does.

Thanks,

Daniel

Re: Repository became corrupt on commit

[Daniel Shahaf]

Daniel Shahaf wrote on Tue, 23 Oct 2018 14:50 +:
> Alan Spark wrote on Tue, 23 Oct 2018 14:41 +0100:
> > perl dump-noderev.pl /path/to/repository /trunk/scripts/script.py 728
> > Use of uninitialized value $noderev_id in split at dump-noderev.pl line 41.
> 
> Sorry about that; the script is obviously missing an error check.
> 
> Looks like 'svnlook' does not exist in your $PATH.
> 
> It's not obvious to non Perl speakers, but line 6 does the
> equivalent of:
> .
> if [ -z "$SVNLOOK" ]; then SVNLOOK=svnlook ; fi
> .
> so if you run the script with svn/svnadmin/svnlook in PATH, _or_ with
> $SVN, $SVNADMIN, $SVNLOOK set to those executables' full paths, then it
> should work.

The above is the preferred way forward, but if you hit roadblocks with
that, there _is_ a crude alternative:

% grep -a '^type:' /path/to/repository/db/revs/0/728

This will extract only one field from the rfc822-formatted node-rev
header.  It's a poor substitute to the Perl script, but better than nothing.

Cheers,

Daniel

Re: Repository became corrupt on commit

[Daniel Shahaf]

Alan Spark wrote on Tue, 23 Oct 2018 14:41 +0100:
> Hi Daniel,
> 
> Sorry, I meant to reply to all. For the benefit of others, here is what I 
> said:
> 
> ---
> Thanks for the response.
> 
> > where dump-noderev.pl is [1]?  The output should be just a few rfc822-like
> > headers specifying the offset and checksum of the file contents 
> > representation
> > ("rep").  One of the headers should be "type: file".
> 
> I'll take a look at this as soon as I can.
> 
> > Sorry, I don't follow.  Can you explain again what the contents of the
> > file is and what's its significance?  Has the server been restarted
> > between the two commit attempts?  Is it svnserve (in which mode,
> > -i/-t/-d/-T/service) or mod_dav_svn (which MPM)?
> 
> The file is a python script. It is just a text file. The first time it
> was committed it corrupted the repository. We rolled back to a backup
> and tried to commit the same file and it again broke the repository.
> Only when we deleted, then added exactly the same file did the problem
> go away. So it was reproducible until we rolled back and deleted the
> file.
> 

Thanks, this is important information.

> I'm afraid I'm new to perl so I may be doing something wrong, but I
> checked out the entire infrastructure-puppet repository and ran the
> command from within
> ~/infrastructure-puppet/modules/rootbin_asf/files/bin to get the
> following output:
> 
> perl dump-noderev.pl /path/to/repository /trunk/scripts/script.py 728
> Use of uninitialized value $noderev_id in split at dump-noderev.pl line 41.

Sorry about that; the script is obviously missing an error check.

Looks like 'svnlook' does not exist in your $PATH.

It's not obvious to non Perl speakers, but line 6 does the
equivalent of:
.
if [ -z "$SVNLOOK" ]; then SVNLOOK=svnlook ; fi
.
so if you run the script with svn/svnadmin/svnlook in PATH, _or_ with
$SVN, $SVNADMIN, $SVNLOOK set to those executables' full paths, then it
should work.

> Is there a simple command that I can run rather than this script?
> 

Not really.  The script dumps an internal data structure (a node-rev is
basically an inode in Subversion's versioned filesystem) and is the easiest way
to do that.  We could probably instrument the C code instead but (a) I don't 
have
that patch ready, (b) you'd then have to rebuild svn.

> I tried to find out what our MPM is but I am not sure. I ran this command:
> 
> /usr/sbin/apache2 -l
> Compiled in modules:
> 
> I'm not sure if that tells you anything but I don't think it is
> prefork or worker.

It doesn't, I'm afraid.  It looks like you're using a *.so MPM but
that's not proof of _which_ MPM it is.

I'm asking about MPM because that ties to svn's internal caches.  (If
you used prefork, the bug's happening twice would rule out intra-process
caches as a cause.)

Cheers,

Daniel

Re: Repository became corrupt on commit

[Daniel Shahaf]

Alan Spark wrote on Fri, 19 Oct 2018 16:04 +0100:
> * Error verifying revision 728.
> svnadmin: E160013: Filesystem path 'trunk/scripts/script.py' is
> neither a file nor a directory

E160013 is SVN_ERR_FS_CORRUPT, but this message is a new one.

Can you share the output of
.
% dump-noderev.pl ${REPO_DIR} /trunk/scripts/script.py 728
.
where dump-noderev.pl is [1]?  The output should be just a few rfc822-like
headers specifying the offset and checksum of the file contents representation
("rep").  One of the headers should be "type: file".

> The content of that file is what had replaced what used to be the trunk
> folder. [...]  Now that we have deleted then re-committed exactly the same
> file, the repository is back to normal and it seems like an unreproducable
> bug at this stage.

Sorry, I don't follow.  Can you explain again what the contents of the
file is and what's its significance?  Has the server been restarted
between the two commit attempts?  Is it svnserve (in which mode,
-i/-t/-d/-T/service) or mod_dav_svn (which MPM)?

Cheers,

Daniel

[1] 
https://github.com/apache/infrastructure-puppet/blob/0a97d8e60798656d856bb0521bee76b24fbe5574/modules/rootbin_asf/files/bin/dump-noderev.pl

Re: ViewVC of Subversion Forbidden

[Daniel Shahaf]

wuzhouhui wrote on Thu, 18 Oct 2018 20:19 +0800:
> Hi,
> 
> When I click 
> Apache's ViewVC instance
> 
> in https://subversion.apache.org/source-code.html, the server told me I 
> have no permission to access /viewvc/subversion/trunk:
> 
> Forbidden
>  
> You don't have permission to access /viewvc/subversion/trunk/ on this server.

The repository is maintained by the infra team at
us...@infra.apache.org; please follow up with them.

Re: svn_client_list() description in API document(svn_client.h)

[Daniel Shahaf]

Yasuhito FUTATSUKI wrote on Thu, Oct 04, 2018 at 19:37:01 +0900:
> In ^/subversion/trunk/subversion/include/svn_client.h@1841567 says about
> svn_client_list():
> > * Similar to svn_client_list2(), but with @a recurse instead of @a depth.
> > * If @a recurse is TRUE, pass #svn_depth_files for @a depth; else
> > * pass #svn_depth_infinity.
> 
> However, ^/subversion/trunk/subversion/libsvn_client/deprecated.c@1822534,
> implementation of svn_client_list() uses macro
> SVN_DEPTH_INFINITY_OR_IMMEDIATES(recurse), which is svn_depth_infinity if
> recurse is TRUE else svn_depth_immediates.
> 
> I'm afraid I think the description in svn_client.h is mistaken, isn't it?

Good catch! Fixed in r1842813.

(I also changed "TRUE" to "FALSE".)

Thanks,

Daniel

Re: svndumpfilter and svnsync?

[Daniel Shahaf]

Ryan Schmidt wrote on Thu, 04 Oct 2018 06:04 -0500:
> On Oct 4, 2018, at 02:32, Chris wrote:
> > I figured using svnsync to get the "cleaned repo" up to date with the 
> > changes on the "live repo", but a note in the svnsync documentation says 
> > "The only commits and revision property modifications that ever occur on 
> > that mirror repository should be those performed by the svnsync tool". Does 
> > that also include this kind of cleanup operation where I remove paths that 
> > don't exist on HEAD?

Yes.  The precondition for running 'svnsync' is that every revision in
the target repository is identical to the corresponding revision in the
source repository.  "Correspondence", in this sense, simply means
numeric equality: r5 must correspond to r5, not to r6 nor to r4.

> > If I should't use svnsync for this, what should I do instead?

You should use svnsync and set the source repository's URL to a URL
that has authz restrictions denying read to the large binary blobs.

That's it.

Cheers,

Daniel

Re: How to Shutdown Svn Server

[Daniel Shahaf]

Mark Phippard wrote on Wed, 03 Oct 2018 10:14 -0400:
> You can either alter the authz rules to remove all write permissions or
> just add a start-commit hook that does an exit 1.

Additionally, if a pre-revprop-change hook exists then it should either
be deleted or be made to exit with a non-zero exit code.

Error codes meaning "This is a working copy, but I can't read it"

[Daniel Shahaf]

I'm looking into a piece of logic for extracting basic information about
a working copy (basically, the output of `svn info` on the the current
directory or on the wc root), and trying to teach it to detect the
situation "This is a working copy, but I can't read it".

What error codes should I look for?  In a few test cases I managed to
get straight `svn info` to report SVN_ERR_WC_UPGRADE_REQUIRED,
SVN_ERR_WC_UNSUPPORTED_FORMAT, and SVN_ERR_WC_CORRUPT on the one hand,
and SVN_ERR_WC_NOT_WORKING_COPY on the other hand, and I'm sure there
are more...

Thanks,

Daniel

Re: Subversion 1.10.2 Checksum (SHA512)

[Daniel Shahaf]

Daniel Shahaf wrote on Sat, 22 Sep 2018 15:28 +:
> Adam Jensen wrote on Sat, 22 Sep 2018 11:12 -0400:
> > On 09/22/2018 10:13 AM, Daniel Shahaf wrote:
> > > One reason for not creating the file in the form 
> > > "${hash}${two_spaces}${filename}"
> > > is that not everyone uses sha512sum from GNU coreutils.  If there were
> > > additional information in the file besides the hash value, people not on
> > > linux would have had to sed it out.
> > 
> > apr and apr-util each do it differently. Consequently, the
> > idiosyncrasies of each will need to explored and handled separately.

This will work unmodified for quite a few forms of .sha512 files.

grep -w "$(sha512sum subversion-1.10.2.tar.bz2 | sed -e 's/ .*//')" 
subversion-1.10.2.tar.bz2.sha512

Re: Subversion 1.10.2 Checksum (SHA512)

[Daniel Shahaf]

Adam Jensen wrote on Sat, 22 Sep 2018 11:12 -0400:
> On 09/22/2018 10:13 AM, Daniel Shahaf wrote:
> > One reason for not creating the file in the form 
> > "${hash}${two_spaces}${filename}"
> > is that not everyone uses sha512sum from GNU coreutils.  If there were
> > additional information in the file besides the hash value, people not on
> > linux would have had to sed it out.
> 
> apr and apr-util each do it differently. Consequently, the
> idiosyncrasies of each will need to explored and handled separately.

The PGP signatures are presented more uniformly, I think.  (The only variable
is the location of the public keys reference file.)

Re: Subversion 1.10.2 Checksum (SHA512)

[Daniel Shahaf]

Branko Čibej wrote on Sat, 22 Sep 2018 16:54 +0200:
> Ah, that would be this one:
> 
> https://www.apache.org/dyn/closer.cgi?path=subversion/subversion-1.10.2.tar.bz2

That still gives me an HTML page, whether I pass the relpath as a query
parameter (as you have) or as PATH_INFO.

Re: Subversion 1.10.2 Checksum (SHA512)

[Daniel Shahaf]

Branko Čibej wrote on Sat, 22 Sep 2018 16:22 +0200:
> On 22.09.2018 16:13, Daniel Shahaf wrote:
> > Please don't download the artifacts from www*.apache.org but from a
> > mirror.  I think there is a redirector CGI somewhere that automatically
> > redirects you to a mirror close to you, but I can't find it :(
> 
> http://subversion.apache.org/download.cgi
> 
> Linked from our main page.

I meant, a CGI that just gives a 302 redirect to the mirror.  Not one
that generates an HTML page that a shell script would need to parse.

The actual CGI lives here: 
https://svn.apache.org/repos/asf/infrastructure/site/trunk/content/dyn/

Cheers,

Daniel

Re: Subversion 1.10.2 Checksum (SHA512)

[Daniel Shahaf]

Daniel Shahaf wrote on Sat, 22 Sep 2018 14:13 +:
> Adam Jensen wrote on Sat, 22 Sep 2018 09:39 -0400:
> > Given the current situation, can anyone think of a shell script blurb to
> > download the files then verify with the hash?
> 
> wget .../subversion-1.10.2.tar.bz2
> wget 
> https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
> sha512sum subversion-1.10.2.tar.bz2 | grep -x -- "$(cat < 
> subversion-1.10.2.tar.bz2.sha512)"

Sorry, change -x to -w.  (Or omit it altogether, really.)

Re: Subversion 1.10.2 Checksum (SHA512)

[Daniel Shahaf]

Adam Jensen wrote on Sat, 22 Sep 2018 09:39 -0400:
> Given the current situation, can anyone think of a shell script blurb to
> download the files then verify with the hash?

wget .../subversion-1.10.2.tar.bz2
wget https://www.apache.org/dist/subversion/subversion-1.10.2.tar.bz2.sha512
sha512sum subversion-1.10.2.tar.bz2 | grep -x -- "$(cat < 
subversion-1.10.2.tar.bz2.sha512)"

> Normally, I would do it with something like this:
> 
> #!/usr/bin/env bash
> FILE=subversion-1.10.2.tar.bz2
> if [ ! -f $FILE ]; then
>   echo "Fetching $FILE"
>   wget http://www-eu.apache.org/dist/subversion/subversion-1.10.2.tar.bz2

Please don't download the artifacts from www*.apache.org but from a
mirror.  I think there is a redirector CGI somewhere that automatically
redirects you to a mirror close to you, but I can't find it :(

(The checksum files, in contrast, are intentionally available *only*
from www*.apache.org.)

> sha512sum --check subversion-1.10.2.tar.bz2.sha512 || exit 1

One reason for not creating the file in the form 
"${hash}${two_spaces}${filename}"
is that not everyone uses sha512sum from GNU coreutils.  If there were
additional information in the file besides the hash value, people not on
linux would have had to sed it out.

Cheers,

Daniel

Re: Inconsistencies/bugs in peg revision parsing and help description

[Daniel Shahaf]

Michael Osipov wrote on Sun, 09 Sep 2018 20:35 +0200:
> I would expect the help output to contain information that target which 
> do not explicitly have [@PEGREV] documented are still subject to peg rev 
> parsing and can cause subtile error messages which cannot be explained 
> via help:
> > export: Create an unversioned copy of a tree.
> > usage: 1. export [-r REV] URL[@PEGREV] [PATH]
> >2. export [-r REV] PATH1[@PEGREV] [PATH2]
> > 
> >   1. Exports a clean directory tree from the repository specified by
> >  URL, at revision REV if it is given, otherwise at HEAD, into
> >  PATH. If PATH is omitted, the last component of the URL is used
> >  for the local directory name.
> > 
> >   2. Exports a clean directory tree from the working copy specified by
> >  PATH1, at revision REV if it is given, otherwise at WORKING, into
> >  PATH2.  If PATH2 is omitted, the last component of the PATH1 is used
> >  for the local directory name. If REV is not specified, all local
> >  changes will be preserved.  Files not under version control will
> >  not be copied.
> > 
> >   If specified, PEGREV determines in which revision the target is first
> >   looked up.
> 
> PATH and PATH2 are not subject to @PEGREV.

I think you have a point.

In «svn export foo bar», 'foo' is a coordinate in the history — a (path,
revision) tuple — and therefore supports peg revisions.  However, that's
not true for 'bar', which is a coordinate in the path space only, and
not even the in-repository path space.  For example, one could easily
imagine an 'svn export' syntax in which 'bar' points into an FTP upload
space, or into a git repository, etc.; specifying peg revisions makes no
more sense for unversioned local filesystem paths than for these examples.

A similar argument holds for «svn mkdir baz».  In that case, 'baz' is a
coordinate in the repository's path space, but it does not yet exist in
the repository's revision space; there is no (path, revision) tuple for
'baz' and hence no natural value to set the peg revision too.

So, I agree, it would have made sense not to parse peg revisions in
these cases.

That's not to say that we should change now, though.  There are
backwards compatibility implications to be considered.

We can, at least, clarify the synopsis.  For example:
.
- usage: 1. export [-r REV] URL[@PEGREV] [PATH]
-2. export [-r REV] PATH1[@PEGREV] [PATH2]
+ usage: 1. export [-r REV] URL[@PEGREV] [UNVERSIONED_PATH[@]]
+2. export [-r REV] WC_PATH[@PEGREV] [UNVERSIONED_PATH[@]]
.
where I made two changes: I added a [@] trailer and renamed the
placeholder arguments.

Cheers,

Daniel

Re: Error connecting to svn.apache.org

[Daniel Shahaf]

Paul Hammant wrote on Sun, 02 Sep 2018 19:30 +:
> I'm not sure if there's a online definitive error guide for Svn, but
> if there is the page for E170013 could be enhanced to suggest "check
> your darn proxy settings in ~/.subversion/servers, fool" (or less
> Paul-centric language).

We should start such a thing on
https://svn.apache.org/repos/asf/subversion/site/publish/ ; the
recent OpenSSL example added to the FAQ and 1.10 release notes can go
there too.



Cheers,

Daniel

Re: Tree conflict resolution considered harmful

[Daniel Shahaf]

Dag-Erling Smørgrav wrote on Thu, 30 Aug 2018 23:49 +0200:
> Daniel Shahaf  writes:
> > Dag-Erling Smørgrav  writes:
> > > % svn resolved $(svn stat | awk '$2 == "C" { print $3 }')
> > 'svn resolved -R ./' is equivalent and more robust.
> 
> No.  My command line only resolves tree conflicts.  Yours also resolves
> text conflicts.

Fair enough.  I don't think we have an easy syntax for resolving only
tree conflicts.

You probably know the following, but for the list's benefit I'll say it:
awk(1) is not a robust way to parse `svn status` output; the output is
columns-oriented, not whitespace-separated-fields oriented.  `svn st |
cut` or `svn st --xml` are the recommended idioms.

Cheers,

Daniel

Re: Tree conflict resolution considered harmful

[Daniel Shahaf]

Stefan Sperling wrote on Thu, 30 Aug 2018 14:06 +0200:
> In --non-interactive mode the default value for --accept is 'recommended'.

This is a backwards incompatible change to the semantics of `svn merge
--non-interactive` (with no other --option flags): A workflow designed
under 1.9 and trusting svn to obey PEP 20's "In the face of ambiguity,
refuse the temptation to guess" will find that 1.10 no longer obeys
that.

I don't know how I missed this before the release, and I'm not sure
what's best to do now, but I wanted to point this out anyway.

Cheers,

Daniel

P.S. If I'd noticed this ahead of the release, I'd probably have suggested
leaving accept=postpone the default and having the output of 'merge' say
"You may want to run 'svn resolve --accept=recommended -R ./' now" at
the end.  (Come to think of it, accept=recommended could/should be the
default for 'svn resolve', couldn't it?)

Re: Tree conflict resolution considered harmful

[Daniel Shahaf]

Dag-Erling Smørgrav wrote on Thu, 30 Aug 2018 14:49 +0200:
> % svn resolved $(svn stat | awk '$2 == "C" { print $3 }')

'svn resolved -R ./' is equivalent and more robust.

Re: Tree conflict resolution considered harmful

[Daniel Shahaf]

Dag-Erling Smørgrav wrote on Wed, Aug 29, 2018 at 12:54:43 +0200:
> Please add an option to disable conflict resolution entirely.

As a hotfix, I think this will do what you need?

Index: subversion/libsvn_client/conflicts.c
===
--- subversion/libsvn_client/conflicts.c(revision 1838961)
+++ subversion/libsvn_client/conflicts.c(working copy)
@@ -11366,6 +11366,7 @@ conflict_type_specific_setup(svn_client_conflict_t
   svn_boolean_t tree_conflicted;
   svn_wc_conflict_action_t incoming_change;
   svn_wc_conflict_reason_t local_change;
+  return SVN_NO_ERROR;
 
   /* For now, we only deal with tree conflicts here. */
   SVN_ERR(svn_client_conflict_get_conflicted(NULL, NULL, &tree_conflicted,

Re: Reintragration using 'swtich'

[Daniel Shahaf]

Anton Shepelev wrote on Wed, 22 Aug 2018 20:03 +0300:
> Is the wroking copy of the branch not required for reintegration?

It's not required.  You can 'switch' your working copy to trunk and run
'svn merge' passing it the URL of the feature branch.

Re: svn_load_dirs doesn't support filenames with @ in

[Daniel Shahaf]

Ryan Schmidt wrote on Tue, 21 Aug 2018 14:16 -0500:
> On Aug 21, 2018, at 05:43, Daniel Shahaf wrote:
> > j...@beniston.com wrote on Tue, 21 Aug 2018 09:40 +0100:
> >> This patch adds @ to the end of the filename, to make it work.
> > 
> > Thanks jon.  Committed with tweaks in r1838539.
> 
> It seems something like this was suggested before:
> 
> https://svn.haxx.se/users/archive-2010-09/0565.shtml
> 
> You directed the contributor to repost the patch to the -dev list, but 
> I'm not sure if they ever did.
> 
> The patch they suggested contained a few additional changes; not sure if 
> they're still relevant now.

Good memory :).  On a skim, the first hunk might still be relevant.
I'll leave it to others to assess this further (Jon, you're welcome to
take this, if you wish).

Cheers,

Daniel

Re: svn_load_dirs doesn't support filenames with @ in

[Daniel Shahaf]

Nico Kadel-Garcia wrote on Tue, 21 Aug 2018 08:39 -0400:
> On Tue, Aug 21, 2018 at 6:43 AM, Daniel Shahaf  
> wrote:
> > j...@beniston.com wrote on Tue, 21 Aug 2018 09:40 +0100:
> > Your MUA seems to have mangled the patch.  The easiest way to avoid this
> > is to send patches as attachments named *.txt.
> 
> Sending attachments for other people to open can get well, it can
> get confusing, and it makes the patch tough for people to review
> without downloading the patch. Is there any compelling reason not to
> do *both*? And, since it's a patch file, not to send it as a
> "filename.patch" attachment?

The reason for advising .txt in preference to .patch is to ensure the
attachment's MIME type will be text/*, which in turn will make the
recipient's MUA show the attachment without the recipient having to take
a manual step.

Re: svn_load_dirs doesn't support filenames with @ in

[Daniel Shahaf]

j...@beniston.com wrote on Tue, 21 Aug 2018 09:40 +0100:
> This patch adds @ to the end of the filename, to make it work.

Thanks jon.  Committed with tweaks in r1838539.

> --- svn_load_dirs.pl.in.old.txt 2018-08-21 09:33:02.557893300 +0100
> 
> +++ svn_load_dirs.pl.in 2018-08-21 09:33:12.339005700 +0100
> 
> @@ -1203,6 +1203,12 @@ while (defined (my $load_dir = &get_next
> 
>  print $handle $property_value;
> 
>  close($handle);

Your MUA seems to have mangled the patch.  The easiest way to avoid this
is to send patches as attachments named *.txt.

Cheers,

Daniel

Re: Revert, but give it a different name?

[Daniel Shahaf]

Nathan Hartman wrote on Tue, 14 Aug 2018 12:36 -0400:
> Suppose I modified filename.c. Can I revert, but instead of throwing away
> my changes, revert to a separate file like filename.c.old and keep my
> modified filename.c for comparison in my favorite tool?

svn cat filename.c@BASE > filename.c.old

but a better way to achieve what you're trying to do is to use a custom 
diff-cmd.

Re: Best way to correct directory error?

[Daniel Shahaf]

Bo Berglund wrote on Sat, 11 Aug 2018 16:41 +0200:
> Now he has put his project sources into:
> /Projectname/trunk/Projectname
> 
> What is the best way to fix this so it will reside in
> /Projectname/trunk instead?
> 
> - Check out /Projectname/trunk and receive subdir Projectname
> - Move to trunk on client
> - svn mv Projectname/* ./*
> - svn ci -m "log message"
> 

This is fine.  Note that * does not usually match dotfiles, so you
should make sure to also 'svn mv Projectname/.foo .foo' for every
dotfile in Projectname/.

There are other ways but in this case there's no need for them.
(They're more useful in the opposite case, when one wants to switch from
a ^/ structure to a ^/trunk/ structure; or if there were
branches of /Projectname/trunk/Projectname around.)

> Or after checkout:
> - Move to trunk/Projectname on client
> - svn mv * ../
> - mv ..
> - svn ci -m "log message"

Assuming the second 'mv' was meant to be a 'cd', that's equivalent.

Cheers,

Daniel

Re: Some files stay at an too new revision when updating the working copy to an old revision

[Daniel Shahaf]

Franz Sirl wrote on Tue, 24 Jul 2018 22:00 +0200:
> It turns out this is probably issue #4700 which for some reason was 
> never approved for the 1.9 branch, even though it was already nominated 
> last October. So the version that matters here is the server version. 
> After I upgraded the server from 1.9.5 to 1.10.2 the problem is gone 
> (regardless of the client version) and all is well now.

Thanks for following up with this information!

Re: subversion-1.10.2/subversion/mod_dav_svn/reports/list.c:273L bad if ?

[Daniel Shahaf]

David Binderman wrote on Sun, 22 Jul 2018 10:18 +:
> subversion-1.10.2/subversion/mod_dav_svn/reports/list.c:273]: (style) 
> Expression is always false because 'else if' condition matches previous 
> condition at line 271.

(I replied on https://issues.apache.org/jira/browse/SVN-4765.  David,
for future reference, we prefer emails to tickets.  Thanks again for
the report.)

Re: using -F fails using bash process substitution. bug?

[Daniel Shahaf]

Philip Martin wrote on Thu, 19 Jul 2018 16:16 +0100:
> We might be able to fix the code that uses stat()
> by having it check for EOF as well.

I see you've now done this in r1836306.  Any reason not to backport
that?  (Feel free to add my +1)

Re: using -F fails using bash process substitution. bug?

[Daniel Shahaf]

Philip Martin wrote on Thu, 19 Jul 2018 23:07 +0100:
> Daniel Shahaf  writes:
> 
> >> The workaround is to use:
> >> 
> >>   echo -ne "ignorethis\n" | svn ps svn:ignore -F - .
> >
> > By the way, an even simpler workaround in this case is
> >
> > svn ps svn:ignore -m $'ignorethis\n' ./
> 
> No.  In this case -F specifies the property value, not the log message.
> 

Ah, my bad.  In that case, just drop the -m:

svn ps svn:ignore $'line1\nline2\nline3\n' ./

This also shows one way to propset a multiline value.

> Something like this may work:
> 
>svn ps svn:ignore $(echo -ne "ignorethis\n") .
> 
> but quoting multiple line values can be tricky.

This should work:

svn ps svn:ignore "$(printf '%s\n' 'line1' 'line2' 'line3' ...)"

(up to trailing newlines)

Cheers,

Daniel

Re: using -F fails using bash process substitution. bug?

[Daniel Shahaf]

Philip Martin wrote on Thu, 19 Jul 2018 16:16 +0100:
> Zing Shishak  writes:
> > The following fails to set svn:ignore using process substitution for the
> > file option:
> >
> > $ svn ps svn:ignore -F <(echo -ne "ignorethis\n") .
> > property 'svn:ignore' set on '.'
> >
> > The property is created but it's empty. Is this something that's not
> > expected to work? I seem to remember doing this in the past and having
> > it work.
> 
> The workaround is to use:
> 
>   echo -ne "ignorethis\n" | svn ps svn:ignore -F - .

By the way, an even simpler workaround in this case is

svn ps svn:ignore -m $'ignorethis\n' ./

Re: using -F fails using bash process substitution. bug?

[Daniel Shahaf]

Philip Martin wrote on Thu, 19 Jul 2018 16:16 +0100:
> It's possible it did work in the past and in other places that construct
> does work, e.g.
> 
>svnmucc -mm put <(echo foo) URL
> 
> Subversion has multiple ways to read a file and when reading from a pipe
> (and that shell command constructs a pipe)

Some shells support a =(...) construct, which is like <(...) except it
expands not to a pipe but to an ordinary file, which can be seek()ed,
stat()ed, etc., so this problem wouldn't arise.

> the code that uses stat() to get the filesize doesn't work because the
> pipe filesize is zero.  There is other code that doesn't use stat() and
> reads in block until EOF, and that works for pipes.  We might be able
> to fix the code that uses stat() by having it check for EOF as well.

Cheers,

Daniel

Re: Subversion server

[Daniel Shahaf]

Daniel Shahaf wrote on Thu, 05 Jul 2018 15:22 +:
> Johan Corveleyn wrote on Thu, 05 Jul 2018 11:53 +0200:
> > I do think it would be a great feature though, to have --include and
> > --exclude options (with glob-like syntax, like our recent --search
> > option for 'svn ls') for certain svn commands. With server-side
> > support to only send the matching items (and falling back to pure
> > client-side filtering when talking to older servers). Then one could
> > do these as an efficient operation (if there is server-side support):
> > 
> 
> Even without any new server-side support, if the client can be assumed
> to have `svn ls -R` output, then it can simply drive the reporter in the
> right way for the server to return exactly the right data.
> 

The "reporter" is the server-side interface that answers 'svn update' requests.

> > * svn cat -R --include '*.txt' $URL
> > * svn export --include 'build.xml' $URL (<- creating sparse directory
> > structure with only the matching files)
> > * svn export --exclude '*.iso' $URL
> 
> This reminds me of the new authz-with-globs feature...

Re: Subversion server

[Daniel Shahaf]

Johan Corveleyn wrote on Thu, 05 Jul 2018 11:53 +0200:
> I do think it would be a great feature though, to have --include and
> --exclude options (with glob-like syntax, like our recent --search
> option for 'svn ls') for certain svn commands. With server-side
> support to only send the matching items (and falling back to pure
> client-side filtering when talking to older servers). Then one could
> do these as an efficient operation (if there is server-side support):
> 

Even without any new server-side support, if the client can be assumed
to have `svn ls -R` output, then it can simply drive the reporter in the
right way for the server to return exactly the right data.

> * svn cat -R --include '*.txt' $URL
> * svn export --include 'build.xml' $URL (<- creating sparse directory
> structure with only the matching files)
> * svn export --exclude '*.iso' $URL

This reminds me of the new authz-with-globs feature...

Re: svn 1.10: mod_authz no longer combines ACL entries

[Daniel Shahaf]

Branko Čibej wrote on Wed, Jul 04, 2018 at 13:34:04 +0200:
> On 04.07.2018 13:25, Michael Ruder wrote:
> > Is this really an intentional change? It results in our case in a huge
> > amount of duplicated ACL entries and seems a rather drastic change
> > regarding backwards compatibility.
> 
> This kind of change is probably not intentional. It does appear that we
> need more test cases, though.

https://svn.apache.org/r1835049

Re: svn st without the X lines for externals?

[Daniel Shahaf]

Nathan Hartman wrote on Sun, 01 Jul 2018 10:31 -0400:
> Thank you for mentioning this. I will look into it and perhaps it can be
> fixed.

Probably just need to make apply_color return its first argument
unmodified when "not sys.stdout.isatty()".

Re: SVN patch issue with property changes

[Daniel Shahaf]

Dipu H wrote on Thu, Jun 28, 2018 at 15:23:35 +:
> It is a conflict. Values didn’t change after applying the patch.
> 
> root@contrail-ubm-dipuh:/tmp/tests# svn patch /tmp/patch
>  Cxpathleak.py
> > rejected hunk ## -0,0 +1,1 ## (svn:executable)
> Summary of conflicts:
>   Property conflicts: 1
> root@contrail-ubm-dipuh:/tmp/tests# svn diff xpathleak.py
> root@contrail-ubm-dipuh:/tmp/tests#

You should probably find all files that have svn:executable set to empty and
change the property's value to "*", and commit that.  That will sidestep the
issue with 'svn patch' modifying an empty property to non-empty that Julian has
described elsethread.

Cheers,

Daniel

Re: SVN patch issue with property changes

[Daniel Shahaf]

Philip Martin wrote on Thu, Jun 28, 2018 at 16:19:30 +0100:
> The normalization of svn:executable values is implemented in the client.
> If some client chooses not to implement that normalization the backend
> will store any value given.
> 
> Should the svn client silently merge '' and '*' for svn:executable
> values or should it be a conflict?  I favour the conflict but I can see
> arguments for a silent merge.

Thinking out loud:

Since the normalization happens in the client, and since the client (evidently)
round-trips unnormalised values unmodified, in 1.11 we could implement
additional values for svn:executable (say, svn:executable=0755 and
svn:executable=0775 to set only the respective unix permission bits); both 1.10
and 1.11 clients would round-trip the new values; 1.11 clients would map the
new values to new semantics and 1.10 clients would map the new values to the
old semantics (0777 &~ umask).

That seems to be an argument in favour of raising a conflict.

Cheers,

Daniel
(It's Friday, though, so I may have made a logical leap of faith along the way… 
☺)

Re: SVN patch issue with property changes

[Daniel Shahaf]

Dipu H wrote on Thu, 28 Jun 2018 14:24 +:
> Looks like svn is not able to properly handle the svn:executable 
> property changes via patch.

Works for me:

[[[
% svnadmin create r
% svn co -q file://`pwd`/r wc 
% cd wc
% touch iota
% svn add -q iota
% svn ci -q -m add 
% svn up -q 
% svn ps -q svn:executable yes iota 
% svn diff > foo 
% svn revert -q iota 
% svn patch -q foo 
% svn diff 
Index: iota
===
--- iota(revision 1)
+++ iota(working copy)

Property changes on: iota
___
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
]]]

You mentioned earlier that svn:executable was present with an empty
value.  Is that really the case?  If it is --- which would be surprising,
as it's supposed to be an impossible state --- it would explain why you
got a conflict.

Re: Unable to shelve changes when using custom diff-cmd

[Daniel Shahaf]

Branko Čibej wrote on Tue, 26 Jun 2018 12:37 +0200:
> It would be helpful if the wrapper script returned the exit code from
> the diff program. Something like 'exit $?' at the end would do it.

Come again?  Appending 'exit $?' to a shell script is a no-op; a script
returns the exit code of its last command.

[[[
% echo '(exit 42)' | sh; echo $?
42
]]]

Re: core dump when specifying revision with trailing dash

[Daniel Shahaf]

Robert Wenner wrote on Mon, 11 Jun 2018 15:56 +0200:
> robert~> svn diff -c 123-
> svn: E235000: In file 'subversion/libsvn_client/ra.c' line 682:
> assertion failed (SVN_IS_VALID_REVNUM(start_revnum))
> Abort trap: 6 (core dumped)
> 
> IMHO Subversion should reject the bad revision number, as it does when I
> specify e.g., "foo" instead.

Thanks for the report.

Yes, an assertion failure is invariably a bug.

I can reproduce that with recentish trunk (that I had lying around) but haven't 
debugged.

Re: Problem setting Notepad++ as log message editor for svn 1.9.7 (Windows7)

[Daniel Shahaf]

Bo Berglund wrote on Fri, 08 Jun 2018 18:29 +0200:
> How can I use Notepad++ as the log editor and not have this happening?

Subversion runs the editor command with system() and expects that when
system() returns, if the exit code is zero then the user has finished
editing the log message.  You should ask the Notepad++ guys how to
invoke it in a way that satisfies this contract.  There may be a command-line
flag you can pass (e.g., I use editor-cmd='gvim -f').

Cheers,

Daniel

Re: SVN access became slow

[Daniel Shahaf]

Stefan Sperling wrote on Wed, 06 Jun 2018 15:18 +0200:
> On Wed, Jun 06, 2018 at 12:37:53PM +0200, Johannes van der Vegt wrote:
> > But... Can I have two processes (IPv4 and IPv6) running on the same port
> > together? And will they access the same database on the driver? That may
> > give trouble if two users connect simultaneously, right?
> 
> No, that is not a problem. Writes to the same repository from
> different processes are always serialized properly. 
> [...]

As Stefan said, having multiple, concurrent readers and writers is 100% 
supported.

At the network level there is no problem either.  The IPv4 and IPv6
stacks are independent; the TCP/IPv4 and TCP/IPv6 sockets happen to use
the same port number, but that's not a problem in any way; it's exactly like
running two TCP/IPv4 listening sockets on different port numbers.

Re: SVN access became slow

[Daniel Shahaf]

Stefan Sperling wrote on Tue, 05 Jun 2018 15:33 +0200:
> If your SVN server has an IPv6 address but does not provide service
> on that address, this is expected behaviour. There is no way for
> the SVN client to know that it should not be trying this address.

Well, connecting to the IPv6 address on TCP port 3690 could return a TCP
RST or an ICMPv6 "Host unreachable" or the like, as opposed to no
response at all (which I presume is what wireshark is showing).

Re: Bug in tools/server-side/svnpubsub/svnwcsub.py

[Daniel Shahaf]

sebb wrote on Fri, 25 May 2018 14:14 +0100:
> The following fails because fname is not defined:
> 
> self.read(fname) [1]
> 
> it should be
> 
> self.read(self.fname)

Thanks for the report.  With that change, the [DEFAULT] section doesn't
get emptied by reload().  Would you like to send a patch that fixes that
as well (in addition to the NameError)?

Replies to dev@, please.

Cheers,

Daniel

Attachments policy (was: Re: Issues)

[Daniel Shahaf]

Ralph Seichter wrote on Thu, 24 May 2018 13:39 +0200:
> Can the mailing list administrators please change the configuration so
> that nobody can send 5-MB-photos to this list? Thank you.

How often has that been happening?

Re: How to check out a project based on commit time rather than rev?

[Daniel Shahaf]

Ryan Schmidt wrote on Fri, 18 May 2018 18:55 -0500:
> There is no need for a separate command because any command that takes a 
> revision number can also accept a date and time. If no time is 
> specified, the time is midnight. Subversion then uses the newest 
> revision that precedes the given date and time.

To be more precise, dates are resolved to revision numbers using binary
search on the revisions space, comparing by the svn:date revprop's
value; that process can give different answers over time (as HEAD
increases) if the sequence of svn:date values isn't monotonically
increasing, which can happen, for instance, when doing 'svnadmin load'
into an existing repository.

Re: Strange libtool errors when try to build pyhton and ruby bindings (1.10.0)

[Daniel Shahaf]

Lev Serebryakov wrote on Fri, 13 Apr 2018 19:39 +0300:
> On 13.04.2018 19:20, Lev Serebryakov wrote:
> >  I'm preparing FreeBSD of subversion-1.10.0 and have very strange errors
> > when try to build bindings.
>   Commands (cc) are missed in both cases. perl and Java bindings don't
> have this problem.

Not an answer to your question, but it would have been nice to catch
this problem during the release candidate phase.  Did you try to build
packages of the 1.10.0-rc* releases?

Cheers,

Daniel

Re: How to disable lz4 support when library is installed in system?

[Daniel Shahaf]

> >  What should I pass to "configure" to PREVENT detection & usage of lz4
> > library installed in system? `--without-lz4' doesn't work.

Yes, I can reproduce this.  '--without-lz4' results in:

configure: lz4 configuration via prefix
checking for LZ4_compress_default in -llz4... yes

which is not the intended behaviour.  The intended behaviour in that case is to 
throw an AC_MSG_ERROR:

  AC_MSG_ERROR([Subversion requires LZ4 >= r129, or use 
--with-lz4=internal])

CCing dev@ for this part.

> --with-lz4=internal  may be what you want.
> 
> From what I can gather lz4 is the future so I would be surprised if you 
> can prevent it’s usage.

In principle, if svn were built without lz4 (which isn't supposed to be
allowed by the stock configure) then it won't be able to read or produce
svndiff2 deltas, which are used on the wire and may be used in FSFS f8
on-disk data.

Re: width of output of 'svn shelves' is too large

[Daniel Shahaf]

wuzhouhui wrote on Sat, Mar 31, 2018 at 18:17:17 +0800:
> 
> > On 31 Mar 2018, at 1:28 AM, Daniel Shahaf  wrote:
> > 
> > wuzhouhui wrote on Mon, Mar 19, 2018 at 19:26:08 +0800:
> >> The width of terminal is 80 column, but the width of output of 'svn 
> >> shelves' is 82
> >> at least, I think 80 is better.
> > 
> > Thanks for testing the pre-releases.
> > 
> >> What do you think?
> > 
> > Agreed in principle, but cannot reproduce the original problem.  With 
> > current
> > trunk, the output for me is 70 columns.  By code inspection, that should be 
> > the
> > case so long as the shelf name is ≤29 characters (plus or minus the length 
> > of
> > the right-hand side description, which can vary by ±5 columns).
> > 
> > What version are you testing with?
> 
> Version: subversion-1.10.0-rc1.tar.bz2
> OS: ubuntu 16.04
> Configure args: ./configure --with-lz4=internal
> 

Okay.  1.10.0-rc2 should be released soon (it's being prepared), but I haven't
the time to test it.

Which reminds me, sorry that we took so long to reply to your original
email.  It must have slipped through the cracks.

> In addition, I found another issue:
> 1. Create a empty file (named 'a') and version it, then commit.
> 2. Add some contents to file 'a'
> 3. svn shelve a a
> Now, the file 'a' will be deleted, that isn't what I want.

I assume this is a use-case we'd like to support, but I'll let the devs who
work on shelving comment on this.  (I've pinged them)

The cause will be that the backend of 'shelve' is patch files, which make no
distinction between an empty file and a non-existent file.  (So to support this
use-case we'll have either to swap backends or to make 'svn patch' smarter; I
believe Julian has considered both options)

Thanks again for your patience.

Daniel

Re: width of output of 'svn shelves' is too large

[Daniel Shahaf]

wuzhouhui wrote on Mon, Mar 19, 2018 at 19:26:08 +0800:
> The width of terminal is 80 column, but the width of output of 'svn shelves' 
> is 82
> at least, I think 80 is better.

Thanks for testing the pre-releases.

> What do you think?

Agreed in principle, but cannot reproduce the original problem.  With current
trunk, the output for me is 70 columns.  By code inspection, that should be the
case so long as the shelf name is ≤29 characters (plus or minus the length of
the right-hand side description, which can vary by ±5 columns).

What version are you testing with?

Re: E130003: The XML response contains invalid XML - Follow-up

[Daniel Shahaf]

Philip Martin wrote on Fri, 16 Mar 2018 13:44 +:
> "NOCERA, ANDY"  writes:
> 
> > I used dump and load to debug the malformed node revision ID.  Here
> > are my steps and what learned.  Looks like the revs' file text: entry
> > has a zero instead of size.  By just editing the size, verify worked.
> > No other change was required.  The question is can we correct this
> > ourselves without a dump and load?
> >
> > db/revs/0/1 /usr/tmp/xrepox/db/revs/0/1
> >   diff db/revs/0/1 /usr/tmp/xrepox/db/revs/0/1
> > 18c18
> > < text: 1 76 48 0 ec69809945c46f2bb74e99a3ff7cd917
> > ---
> > > text: 1 76 48 48 ec69809945c46f2bb74e99a3ff7cd917
> > 22c22
> 
> That looks like issue 4554
> 
> https://issues.apache.org/jira/projects/SVN/issues/SVN-4554
> 
> Editing the file is unlikely to work. Later revisons refer to data in
> earlier revisions via a byte offset into the earlier file.  When you
> edit "0" to "48" you have changed the byte offset of all the data beyond
> the edit and that breaks the references in all the later files.

Changing "0" to "48" would also have broken the  and
 offsets in that revision file, so how come 'verify' worked after
that change?

Cheers,

Daniel
(The  terms are from the 'structure' file and describe
format 6; format 7 has  and  which are
analogous)

fsfs 'structure' terminology: length/size v. size/expanded-size (was: Re: E130003: The XML response contains invalid XML - Follow-up)

[Daniel Shahaf]

[ cc += dev@ ]

NOCERA, ANDY wrote on Thu, 15 Mar 2018 22:35 +:
> Folks, 
> 
> I used dump and load to debug the malformed node revision ID.  Here are 
> my steps and what  learned.   Looks like the revs' file text: entry has 
> a zero instead of size.

To be clear, you mean the fourth field.  On line 562 'size' is the name
of the fourth field but on line 119, and in the representation_t struct,
'size' is the name of the third field.[1]

This is a recipe for confusion.  Normally I think of the fields using
the size/expanded-size terminology so I propose to change 'structure' to
use that terminology in lieu of length/size (and leave a note behind
pointing out that old versions of the document used  differently).

WDYT?

Daniel

[1] All line numbers refer to subversion/libsvn_fs_fs/structure.

>   diff db/revs/0/1 /usr/tmp/xrepox/db/revs/0/1
>   18c18
>   < text: 1 76 48 0 ec69809945c46f2bb74e99a3ff7cd917
>   ---
>   > text: 1 76 48 48 ec69809945c46f2bb74e99a3ff7cd917
>   22c22

Re: Can import also convert the source to a working copy

[Daniel Shahaf]

Branko Čibej wrote on Thu, 15 Mar 2018 21:37 +0100:
> On 15.03.2018 20:18, Bo Berglund wrote:
> > On Thu, 15 Mar 2018 13:45:13 -0400, Kris Deugau 
> > wrote:
> >
> >> Unless I misread your original post, the very first option in that link 
> >> looks like a better fit.  To rephrase it somewhat for your case:
> >>
> >> 1)  Create a repository or a directory in the repository
> >> 2)  Create your local files
> >> 3)  Check out the empty repository path to your workspace - this won't 
> >> overwrite any of your files
> >> 4)  svn add [files]
> >> 5)  svn ci
> >> 6)  Continue working as usual
> >>
> >> This avoids a round trip to the server to push the current files, then 
> >> pull them back down to create the formal SVN working copy with things 
> >> already in it - instead you "check out" an empty directory which should 
> >> be quite fast.
> >>
> > I readthat page as best I could but it looked so much Linuxish
> 
> What on earth is Linuxish about it?

You don't have to use the 'svn' client; you can use any Subversion
client (a GUI client, TortoiseSVN, IDE integrations...).

The docs always use the 'svn' client because (1) it's the only non-third-
party client, (2) it's a command-line client so it's easier to give
instructions for it.

Re: Run shelve tests in subversion 1.9

[Daniel Shahaf]

wuzhouhui wrote on Sun, 11 Mar 2018 19:51 +0800:
> > On 11 Mar 2018, at 7:15 PM, wuzhouhui <1530108...@qq.com> wrote:
> >> On 11 Mar 2018, at 6:29 PM, Branko Čibej  wrote:
> >>> On 11.03.2018 11:20, wuzhouhui wrote:
> >>> I am backporting 'svn shelve' subcommand into Subversion 1.9,
> >>> and now I want
> >>> to do some tests. If I copy shelve_tests.py from Subversion-1.10 to
> >>> Subversion-1.9/subversion/tests/cmdline/shelve_tests.py, and run 'make 
> >>> check',
> >>> but shelve_tests.py doesn't executed.
> >>> 
> >>> So my question is how to run shelve_tests.py in Subversion 1.9 when 'make
> >>> check'?
> >> 
> >> Most likely you forgot to make the file executable.
> The file is executable.

What happens when you run 'python2 ./shelve_tests.py'?

(I don't think our infrastructure cares whether the +x bit is set.)

Re: SHA-1 collision in repository?

[Daniel Shahaf]

Daniel Shahaf wrote on Fri, Mar 02, 2018 at 22:57:51 +:
> Myria wrote on Mon, Feb 26, 2018 at 13:41:05 -0800:
> > In other news, unknown whether related to the current problem, my
> > attempt to clone the repository to my local computer is failing:
> > 
> > D:\>svnsync sync file:///d:/svnclone
> > Transmitting file data
> > .svnsync:
> > E16: SHA1 of reps '227170 153 193 57465
> > bb52be764a04d511ebb06e1889910dcf
> > e6291ab119036eb783d0136afccdb3b445867364 227184-4vap/_4o' and '-1 0
> > 193 57465 bb52be764a04d511ebb06e1889910dcf
> > e6291ab119036eb783d0136afccdb3b445867364 227184-4vap/_4o' matches
> > (e6291ab119036eb783d0136afccdb3b445867364) but contents differ
> > svnsync: E160004: Filesystem is corrupt
> > svnsync: E200014: Checksum mismatch while reading representation:
> >expected:  bb52be764a04d511ebb06e1889910dcf
> >  actual:  80a10d37de91cadc604ba30e379651b3
> 
> When this error happens, could you print the first lines of the two reps
> identical?  The first line is "PLAIN\n" or "DELTA\n" or "DELTA 42 43 44\n".
> (I wonder whether we have some stray whitespace that's transparent to parsing
> but breaks checksums.)

In second thought I'm not sure this makes sense.  A better question is: can we
obtain the fulltext whose checksum is 80a10d37de91cadc604ba30e379651b3?

> Do you happen to have a copy of the repository lying around that you can run
> 'grep -a 80a10d37de91cadc604ba30e379651b3 db/revs/{0,1,2,...,227}' on?
> Admittedly that's a bit of a shot in the dark.
> 
> Cheers,
> 
> Daniel

Re: SHA-1 collision in repository?

[Daniel Shahaf]

Myria wrote on Thu, Mar 01, 2018 at 18:45:38 -0800:
> Also, I have no control over what was in the repository five years
> ago.  The huge files were compiled versions of WebKit libraries.

Note that in 2017, WebKit intentionally committed a SHA-1 collision into their
repository.  If you have the WebKit sources in your tree too, that'd be 
something
to watch out for, but it can't explain the r227185 instance.

Cheers,

Daniel

Re: SHA-1 collision in repository?

[Daniel Shahaf]

Myria wrote on Mon, Feb 26, 2018 at 13:41:05 -0800:
> In other news, unknown whether related to the current problem, my
> attempt to clone the repository to my local computer is failing:
> 
> D:\>svnsync sync file:///d:/svnclone
> Transmitting file data
> .svnsync:
> E16: SHA1 of reps '227170 153 193 57465
> bb52be764a04d511ebb06e1889910dcf
> e6291ab119036eb783d0136afccdb3b445867364 227184-4vap/_4o' and '-1 0
> 193 57465 bb52be764a04d511ebb06e1889910dcf
> e6291ab119036eb783d0136afccdb3b445867364 227184-4vap/_4o' matches
> (e6291ab119036eb783d0136afccdb3b445867364) but contents differ
> svnsync: E160004: Filesystem is corrupt
> svnsync: E200014: Checksum mismatch while reading representation:
>expected:  bb52be764a04d511ebb06e1889910dcf
>  actual:  80a10d37de91cadc604ba30e379651b3

When this error happens, could you print the first lines of the two reps
identical?  The first line is "PLAIN\n" or "DELTA\n" or "DELTA 42 43 44\n".
(I wonder whether we have some stray whitespace that's transparent to parsing
but breaks checksums.)

Do you happen to have a copy of the repository lying around that you can run
'grep -a 80a10d37de91cadc604ba30e379651b3 db/revs/{0,1,2,...,227}' on?
Admittedly that's a bit of a shot in the dark.

Cheers,

Daniel

Re: Reverting accidental permanent acceptance of SSL certificate?

[Daniel Shahaf]

Bo Berglund wrote on Sun, 11 Feb 2018 21:00 +0100:
> I found the certificate acceptance in this file:
> 
> ~/.subversion/auth/svn.ssl.server/52e60f46d8c02303aea5256b18eb7aac
> 
> It looks perfectly safe, does not contain anything especially useful.

That file records the certificate itself (in standard encoding) and the
list of failures the certificate had when you permanently accepted it
(such as having expired, being self-signed, etc).  If when the certificate
is next seen it has only those failures that it had last time, it won't be
prompted for again.

Re: EOL problems in cvs2svn converted repository

[Daniel Shahaf]

Bo Berglund wrote on Sun, 11 Feb 2018 09:34 +0100:
> For OpenVPN I found that I had to add some servers to apt-get in order
> to use the OpenVPN own repository, is there a similar solution for
> Subversion?

See https://subversion.apache.org/packages

... but unless there is a specific bugfix from 1.9.4..1.9.7 that you
need, 1.9.3 should be fine.  See
https://svn.apache.org/repos/asf/subversion/trunk/CHANGES
(that file includes information for all past releases and some future ones)

Re: EOL problems in cvs2svn converted repository

[Daniel Shahaf]

Johan Corveleyn wrote on Fri, 09 Feb 2018 13:40 +0100:
> So changing this in historic revisions (in a dump file) is not just a
> question of adding the property, it also involves normalizing the
> corresponding file contents in the dump (and the dump format contains
> checksums and offsets, so manipulating actual contents in not easy).

Why wouldn't it be easy?

If the dumpstream was generated without --deltas, the records are not
dependent on each other, so writing some sort of stdin-to-stdout filter
that renormalizes newlines of some file records would be
straightforward.  It wouldn't be doable with sed, for the reasons you
give, but it'd be doable using the API.

If the dumpstream was generated with --deltas, it would be easier to load
the dumpstream into a repository and then to use svnsync to renormalize
the newlines.

(Conceptually, 'svnsync sync' should allow the source to be a
dumpstream, reusing the dumpstream-to-editor backend of 'svnrdump load',
but that hasn't been implemented.)

Cheers,

Daniel

Re: svn diff non-temporary files option

[Daniel Shahaf]

Branko Čibej wrote on Wed, 07 Feb 2018 14:21 +0100:
> No; because 'svn diff' hides differences in end-of-line style and
> keyword expansions, it has to preprocess the files in the working copy,
> and it sends those preprocessed files to the external diff wrapper.
> 
> We could theoretically skip that part for files that don't have
> svn:eol-style or svn:keywords properties, but then the tool invocation
> would be inconsistent.
> 
> Your best bet is to use an editor that has Subversion support built in.

Or we could add some context information in argv[8] of the diff-cmd to
make it easier for them to be intelligent.  (Right now only 1..7 are used, IIRC)

Re: Building 1.8.19 from source tarball

[Daniel Shahaf]

Stefan Sperling wrote on Wed, 31 Jan 2018 16:58 +0100:
> On Wed, Jan 31, 2018 at 10:45:07AM -0500, Nathan Hartman wrote:
> > The get-deps.sh script that comes with the source tarball is fetching
> > apr-1.4.6 and apr-util-1.5.1. There are newer versions available,
> > namely apr-1.6.3 and apr-util-1.6.1.
> 
> You should be able to use current APR releases without issues.

We don't update the get-deps.sh recommended version whenever APR makes
a release; we only update that in the run-up to a 1.x.0 release and when
upstream APR has a high-severity bug.  Consequently, it's common that
even on the day a Subversion release is made, its get-deps.sh wouldn't point to
the latest APR version, but to whichever was latest at the time of the 
preceding 1.x.0.

Re: Searching for a C++ API

[Daniel Shahaf]

R developer wrote on Tue, 30 Jan 2018 10:22 +0100:
> I'm not that familiar with the C API unless I'm missing something
> (which is very well possible) one would have to manage the apr pools
> himself?

In the C API, the caller of a function passes to it a result_pool and a
scratch_pool, and decides when to reclaim or free their memory space
(apr_pool_clear / apr_pool_destroy).

> for instance
> https://svn.apache.org/repos/asf/subversion/trunk/tools/examples/minimal_client.c
> does seem very complex to me for the tasks it performs is there any
> easier C example out there?

svnlook? svnmucc? subversion/tests/cmdline/*.c?

There were C++ language bindings developed in trunk, see
subversion/bindings/cxxhl/.  I don't know what the state of the art on
that is --- quite possibly they aren't stable yet --- but they are
Apache Licensed, so perhaps you could build on them.  If they don't
yet provide some functionality which you require, please do send
patches :-).

Cheers,

Daniel