Re: Changing the permission checks in libsvn_subr/io.c

[Vincent Lefevre]

On 2024-01-22 04:05:04 +0100, Branko Čibej wrote:
> On 13.01.2024 09:58, Daniel Sahlberg wrote:
> > Since there wasn't any replies to this and I think the code was working
> > fine in all my tests, I comitted as r1915214. Although I finally decided
> > to solve the spurious revert messages in a different way, see a separate
> > followup/committ e-mail.
> 
> I think you just made the code more complicated for no good reason. The
> situation you described and fixed applies only to working copies that are
> shared by different users.

Note that in my case, the working copy was not shared by different
users. It just happened that some files had a different owner (as
I had done a "cp" under root by mistake).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Changing the permission checks in libsvn_subr/io.c

[Vincent Lefevre]

On 2024-01-10 09:44:51 +0100, Johan Corveleyn wrote:
> Interesting discussion. I agree it should at least be documented, and
> perhaps be made a bit more clear from the output of 'revert' (but not
> sure how far we can go without breaking compat). Changing the current
> behavior is probably a more risky move, given the maturity of SVN and
> backwards compatibility etc.

Another solution would be to make the behavior configurable
(in .subversion/config).

[...]
> BTW, I know about a similar issue of 'spurious revert notifications',
> with mismatching timestamps (lastmod-time different from svn metadata,
> a condition that is normally fixed by 'svn cleanup'): if you "touch" a
> file in your WC so it has a different timestamp from the metadata, it
> will be notified when running 'svn revert' (and I believe the metadata
> is adjusted in this case). So that's another source of spurious revert
> notifications. I thought I had discussed this on one of our
> mailinglists, but I can't find it. The fact that revert fixes
> timestamps is mentioned in passing in issue SVN-4162.

But there are no properties related to the behavior concerning
timestamps (a "touch" may be done on purpose for timestamp-based
utilities like "make").

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Changing the permission checks in libsvn_subr/io.c

[Vincent Lefevre]

On 2024-01-05 11:29:16 +0100, Daniel Sahlberg wrote:
> Den fre 5 jan. 2024 kl 10:51 skrev Johan Corveleyn :
> 
> > On Fri, Jan 5, 2024 at 8:46 AM Daniel Sahlberg
> >  wrote:
> > ...
> > > Since the file doesn't have svn:needs-lock it should be RW [and the
> > Reverted message comes from Subversion trying to restore the W flag ...]
> >
> > Should it? Intuitively I'd say: since the file doesn't have
> > svn:needs-lock Subversion shouldn't be looking at R or RW. Why should
> > we make a file RW? Can't the user make a file readonly just locally,
> > and expect Subversion not to care?
> >
> > Or is "making a file readonly" a committable local change? Will it
> > show up on 'svn st' and can it be committed as some change that can be
> > transferred to another working copy?
> >
> > I understand that svn:needs-lock adds extra handling of the readonly
> > status of files, but without that property?
> 
> All good questions, and I probably agree with you: if svn:needs-lock isn't
> set then Subversion could just ignore the R/RW status.

I also agree. I never use svn:needs-lock, and I want to be able to
set some files in my working copy to read-only, in order to make sure
that I won't modify them by mistake.

> But any change here would change previous behaviour so it would need
> a solid consensus.

In any case, the current behavior of Subversion is inconsistent.
"svn revert" is not documented as the command to "fix" the permissions.

> If the check is removed for files that doesn't svn:needs-lock, then we
> might have to add code to restore RW status if svn:needs-lock is removed.
> 
> Making a file readonly is currently not a committable change, didn't check
> 'svn st' but it will be reverted by 'svn revert' and it will not be
> transferred to another WC. It can only be committed indirectly via
> svn:needs-lock.
> 
> Any discussion regarding svn:needs-lock probably also have to consider
> svn:executable, since it is handled similarly (except on WIN32 and OS2,
> where the concept of executable doesn't exists).
> 
> I havn't completely made up my mind, but I think I favour keeping the
> current behaviour: R/RW status in indicated by the svn:needs-lock property
> and you shouldn't change R/RW manually within a WC.

Then this should be documented.

But "svn st" should detect and report incorrect permissions, and
"svn up" should fix them, just like what happens when a file has
been removed with just "rm" instead of "svn delete".

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn diff --diff-cmd" + script using /usr/bin/env broken with Android 14

[Vincent Lefevre]

Hi,

This is a bit old, but this is apparently a Termux specific issue,
which has just been fixed (but the fix isn't available yet, so that
I couldn't test).

On 2023-11-20 03:43:31 +0900, Yasuhito FUTATSUKI wrote:
> Hello,
> 
> On 2023/11/19 11:34, Vincent Lefevre wrote:
> > I've reported the following bug on
> > 
> >https://github.com/termux/termux-packages/issues/18537
> > 
> > Since this is specific to svn, I'm reporting it here too.
> > In particular, I'm wondering what special thing svn does
> > to trigger the failure compared to the shell.
> > 
> > With "svn diff", I use the --diff-cmd option with my own diff utility,
> > which is a script that runs an interpreter using /usr/bin/env. I use
> > this config on various machines, including on my Android phones in
> > Termux. After the upgrade to Android 14, this is no longer working.
> 
> "svn diff --diff-cmd ..." on Unix platforms uses execvp(3) through
> svn_io_run_diff2(), svn_io_run_cmd() (inherit == TRUE),
> svn_io_start_cmd3(), and apr_proc_create(), and no special
> treatment about environment variables.

The execvp() function wasn't wrapped by Termux to fix paths. It seems
that the issue is that /usr/bin/env doesn't exist, hence the need for
a wrapper to use the actual pathname. However, this doesn't explain
why the standard execvp() was working with Android 13.

> [[[
> #!/bin/sh
> if [ ! -n ${ZSH_VERSION} ]; then
> exec zsh "$0" "$@"
> fi
> ...
> ]]]

I'm currently using

#!/bin/sh
[ -n "$ZSH_VERSION" ] || exec zsh -f -- "$0" ${1+"$@"}

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Fix issue SVN-4622 (Was: Re: "svn revert -R ." outputs spurious Reverted messages)

[Vincent Lefevre]

On 2023-12-26 16:30:21 +0100, Daniel Sahlberg wrote:
> TLDR; svn status is verifying read-only status for each file (based on
> svn:needs-lock).

I never use the svn:needs-lock property. So I don't see why svn
tries to modify the read-only status in this case.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn revert -R ." outputs spurious Reverted messages

[Vincent Lefevre]

On 2023-12-23 22:44:51 +0100, Daniel Sahlberg wrote:
> Thank you for your report and also for finding the root cause. I can
> confirm this is the same on my machine. I'd like to take a closer look but
> I'm not quite sure when I will find the time.
> 
> This has been reported before:
> https://issues.apache.org/jira/browse/SVN-4622

Indeed. Not for the original bug report, but what is said in the
comments.

I'm wondering why svn wants to change the permissions, at least if
there is nothing to revert for the concerned file. I would see this
as a (separate, but related) bug. FYI, in my working copies, I set
some files as read-only in order to make sure I won't modify them
by mistake (even though there would be no data loss with a revert,
except the timestamp). So I think that this is a bad behavior if
"svn revert" changes the write permission.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn revert -R ." outputs spurious Reverted messages

[Vincent Lefevre]

On 2023-12-23 03:49:04 +0100, Vincent Lefevre wrote:
> In one of my working copies:
> 
> qaa% svn st 
> qaa% svn revert -R .
> Reverted 'etc/apache2/conf-available/javascript-common.conf'
> Reverted 'etc/apache2/mods-available/dnssd.load'
> Reverted 'etc/apache2/mods-available/dnssd.conf'
> qaa% svn revert -R .
> Reverted 'etc/apache2/mods-available/dnssd.load'
> Reverted 'etc/apache2/mods-available/dnssd.conf'
> Reverted 'etc/apache2/conf-available/javascript-common.conf'
[...]

I've eventually found the cause: these files has owner root.
Perhaps a mistake I did when I copied the files (but I am
wondering how I could have done that). I've now fixed the
ownership and the messages have disappeared.

However, I don't see why I get such Reverted messages in the case
of an incorrect owner.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

"svn revert -R ." outputs spurious Reverted messages

[Vincent Lefevre]

In one of my working copies:

qaa% svn st 
qaa% svn revert -R .
Reverted 'etc/apache2/conf-available/javascript-common.conf'
Reverted 'etc/apache2/mods-available/dnssd.load'
Reverted 'etc/apache2/mods-available/dnssd.conf'
qaa% svn revert -R .
Reverted 'etc/apache2/mods-available/dnssd.load'
Reverted 'etc/apache2/mods-available/dnssd.conf'
Reverted 'etc/apache2/conf-available/javascript-common.conf'
qaa% svn revert -R .
Reverted 'etc/apache2/mods-available/dnssd.conf'
Reverted 'etc/apache2/mods-available/dnssd.load'
Reverted 'etc/apache2/conf-available/javascript-common.conf'
qaa% svn revert -R .
Reverted 'etc/apache2/conf-available/javascript-common.conf'
Reverted 'etc/apache2/mods-available/dnssd.load'
Reverted 'etc/apache2/mods-available/dnssd.conf'
qaa% svn revert -R .
Reverted 'etc/apache2/mods-available/dnssd.load'
Reverted 'etc/apache2/mods-available/dnssd.conf'
Reverted 'etc/apache2/conf-available/javascript-common.conf'

This occurs with the subversion 1.14.2-5+b1 Debian package under
Debian/unstable on one machine, but not on another one.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

"svn diff --diff-cmd" + script using /usr/bin/env broken with Android 14

[Vincent Lefevre]

I've reported the following bug on

  https://github.com/termux/termux-packages/issues/18537

Since this is specific to svn, I'm reporting it here too.
In particular, I'm wondering what special thing svn does
to trigger the failure compared to the shell.

With "svn diff", I use the --diff-cmd option with my own diff utility,
which is a script that runs an interpreter using /usr/bin/env. I use
this config on various machines, including on my Android phones in
Termux. After the upgrade to Android 14, this is no longer working.

To reproduce:

1. In ~/bin, add a "mydiff" script that starts with "#!/usr/bin/env sh"
   (for this test, it doesn't need to do anything, i.e. it may contain
   just this line).

2. In a Subversion working copy, modify a file (in order to have an
   non-empty diff).

3. Run "svn diff --diff-cmd mydiff".

I get an error like

Index: perso/TODO
===
exec of 'mydiff' failed: No such file or directorysvn: E200012: 'mydiff' 
returned 255

i.e. the execution of "mydiff" fails.

Note that running "mydiff" from the shell does not give an error.
The issue occurs only with "svn diff --diff-cmd mydiff".

And if "mydiff" starts with "#!/bin/sh", then
"svn diff --diff-cmd mydiff" works fine.

Note about the use of "/usr/bin/env": with the actual script, this is
actually "#!/usr/bin/env zsh" with the goal of using the same script
on various platforms (where zsh can be at various places).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [BUG] svn tries to read a directory on a different filesystem and hangs

[Vincent Lefevre]

On 2022-11-06 04:59:19 +0100, Vincent Lefevre wrote:
> I'm also wondering of the consequence of symlinks .svn/entries to
> /path-to-attacked-user/.svn/entries, etc. except for the pristine
> subdirectory, which Mallory creates as world-writable. If the user
> does a "svn up", could this populate the pristine subdirectory
> (owned by Mallory)?

In case this is not clear:

Mallory owns the "/home/.svn" directory.
The user's home is "/home/user".
The user has a working copy "/home/user/private-wc" and Mallory knows
that (or at least can guess).
The symlinks would be:
  /home/.svn/entries -> /home/user/private-wc/.svn/entries
and ditto for "format", "tmp", "wc.db", "wc.db-journal".

Actually this doesn't seem to work as svn expects .svn-base files
(with a test using ~/software/test/.svn instead of /home/.svn):

zira:~/software/test> svn up
svn: E155009: Failed to run the WC DB work queue associated with 
'/home/vinc17/software/test/README', work item 2510 (file-install README 1 0 1 
1)
svn: E02: Can't open file 
'/home/vinc17/software/test/.svn/pristine/41/412b65baed0c5ba451a7151c8630c135b6116557.svn-base':
 No such file or directory

However, this modifies the user's working copy (which shouldn't have
been touched), as "svn up" gives from this working copy:

svn: E155037: Previous operation has not finished; run 'cleanup' if it was 
interrupted

This is bad.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [BUG] svn tries to read a directory on a different filesystem and hangs

[Vincent Lefevre]

On 2022-10-31 10:02:14 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Mon, 24 Oct 2022 13:57 +00:00:
> > "svn" goes up in the directory hierarchy to look for a .svn directory.
> > The issue is that it doesn't stop at filesystem and/or owner change.
> 
> Why should the upwards scan stop at mount points?  Because accessing
> /home/.svn on a random machine in your lab hangs?  That's insufficient
> justification.

There's also a potential security issue if the owner is different,
and that basically the same as below:

> Why should the upwards scan stop at owner change?  What's the facts of
> the setup (a concrete example with relevant ownerships and permissions
> specified) and what could Mallory do that he shouldn't be able to?  Feel
> free to reply on security@ if the matter isn't suitable for public
> discussion.

I don't have to do much testing at the moment, but some ideas...

Mallory could set up a .svn directory he controls. So if one does
"svn info", that would output a last changed author that Mallory
chose. Now, does the svn client check whether there are non-printable
characters in the author name? If it doesn't, this could send escape
sequences to the terminal.

I'm also wondering of the consequence of symlinks .svn/entries to
/path-to-attacked-user/.svn/entries, etc. except for the pristine
subdirectory, which Mallory creates as world-writable. If the user
does a "svn up", could this populate the pristine subdirectory
(owned by Mallory)?

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[BUG] svn tries to read a directory on a different filesystem and hangs

[Vincent Lefevre]

"svn" goes up in the directory hierarchy to look for a .svn directory.
The issue is that it doesn't stop at filesystem and/or owner change.
This has several consequences:

* A potential security issue, because some .svn directory may be
  under control of another user.

* On some machine at my lab (Debian/stable), this makes svn hang
  when trying to open "/home/.svn", which is the home dir of the
  user ".svn" (FYI, emacs tries to get the svn status of a file
  when opening it).

This is reproducible with

svn, version 1.14.2 (r1899510)
   compiled Oct 20 2022, 08:12:24 on x86_64-pc-linux-gnu

under Debian/unstable.

On the Debian/stable machine, this issue is made worse by the fact
that svn still goes up after a svn working copy has been reached:

patate:~/private/backup> svn info

hangs, but not

patate:~/private> svn info
svn: E155036: Please see the 'svn upgrade' command
svn: E155036: The working copy at '/home/vlefevre/private'
is too old (format 9) to work with client version '1.14.1 (r1886195)' (expects 
format 31). You need to upgrade the working copy first.

which fails immediately (this was probably a very old svn working copy,
which I no longer use).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [BUG] "svn propedit" loses changes in case of a network failure

[Vincent Lefevre]

On 2022-09-19 11:02:05 +0900, Yasuhito FUTATSUKI wrote:
> I can't determine whether this behavior is a bug or not. 
> 
> As svn_cl__propedit() calls svn_cmdline__edit_string_externally()[1]
> for editing revprop value string with the parameter tempfail_reft is
> NULL[2], the temporary file is removed within the function
> svn_cmdline__edit_string_externally(), before trying to set the new
> value.  So, it can be reproduced in any scheme, even "file" scheme,
> by rejecting the change of revprop values in pre-revprop-change
> hook.

Well, if the pre-revprop-change hook rejects the change, this may
not be a bug, though I think that the new file should really be kept
somewhere. AFAIK, the svn-commit.tmp file (log message) is kept for
a rejected commit, so I don't see why this should be different.

But here, this is not a rejection due to a hook, but a failure due to
some external cause, on which the user has no control (contrary to a
hook).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [BUG] "svn propedit" loses changes in case of a network failure

[Vincent Lefevre]

On 2022-09-18 01:59:35 +0200, Vincent Lefevre wrote:
> On 2022-09-18 01:41:38 +0200, Vincent Lefevre wrote:
> > With svn 1.14.2 under Debian/unstable, I wanted to edit a log message
> > with
> > 
> >   svn pe --revprop svn:log -r 151946
> > 
> > (not just a minor change, I was replacing text by a much longer text),
> > but got an immediate error from SSH:
> 
> Note: after quitting the editor.

The issue can be reproduced by creating a wrapper script to ssh and
using

  SVN_SSH=/path/to/this/script svn propedit ...

Before quitting the editor, change this script to "exit 1" in order
to simulate a ssh failure. The /tmp/svn-prop.tmp file the user has
modified is no longer there, and there does not seem to be a backup
anywhere else (the error message doesn't tell anyway).

Note: I'm using the "svn+ssh" scheme. I don't know whether this bug
also occurs with the https scheme (I cannot try).

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [BUG] "svn propedit" loses changes in case of a network failure

[Vincent Lefevre]

On 2022-09-18 01:41:38 +0200, Vincent Lefevre wrote:
> With svn 1.14.2 under Debian/unstable, I wanted to edit a log message
> with
> 
>   svn pe --revprop svn:log -r 151946
> 
> (not just a minor change, I was replacing text by a much longer text),
> but got an immediate error from SSH:

Note: after quitting the editor.

FYI, this means that Subversion is vulnerable to a remote attack.
Here are the details. The logs of my server show:

Sep 18 01:24:09 joooj sshd[141287]: error: kex_exchange_identification: 
Connection closed by remote host
Sep 18 01:24:09 joooj sshd[141287]: Connection closed by 197.5.145.64 port 58377
Sep 18 01:24:10 joooj sshd[615]: error: beginning MaxStartups throttling
Sep 18 01:24:10 joooj sshd[615]: drop connection #10 from [197.5.145.64]:58387 
on [155.133.131.76]:22 past MaxStartups

This "beginning MaxStartups throttling" was due to 197.5.145.64 only,
but this means that all the other IP addresses that attempt to
connect are concerned. There is protection by fail2ban, but it works
by looking at the logs, meaning that it takes a few seconds to react:

2022-09-18 01:24:11,513 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,514 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,539 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,540 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,568 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,569 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,569 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,592 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,592 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,608 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,636 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,663 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,697 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,698 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,698 fail2ban.actions[603]: NOTICE  [sshd] Ban 
197.5.145.64
2022-09-18 01:24:11,742 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,963 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,966 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:11,966 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:11
2022-09-18 01:24:13,972 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,972 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,973 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,974 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,975 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,975 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,978 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,979 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,981 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,982 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,983 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,984 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,985 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,986 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,987 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.64 - 2022-09-18 01:24:13
2022-09-18 01:24:13,988 fail2ban.filter [603]: INFO[sshd] Found 
197.5.145.6

[BUG] "svn propedit" loses changes in case of a network failure

[Vincent Lefevre]

With svn 1.14.2 under Debian/unstable, I wanted to edit a log message
with

  svn pe --revprop svn:log -r 151946

(not just a minor change, I was replacing text by a much longer text),
but got an immediate error from SSH:

kex_exchange_identification: read: Connection reset by peer
Connection reset by 155.133.131.76 port 22
svn: E170013: Unable to connect to a repository at URL 'svn+ssh://mysvn'
svn: E210002: To better debug SSH connection problems, remove the -q option 
from 'ssh' in the [tunnels] section of your Subversion configuration file.
svn: E210002: Network connection closed unexpectedly

Subversion apparently does not keep a copy of the text (contrary to
the case of a commit, which leaves a svn-commit.tmp file), so the
whole new text was lost!!!

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: build/generator/templates/*.ezt and .bat files and svn:eol-style

[Vincent Lefevre]

On 2022-08-20 14:08:43 +0200, Branko Čibej wrote:
> On 20.08.2022 00:57, Vincent Lefevre wrote:
> > A minor issue I had found during my checks for the URL update:
> [...]
> > Shouldn't build/generator/templates/vcnet_vsprops.ezt have
> > svn:eol-style set to native so that it doesn't get CRLF
> > line terminators under Unix?
> > 
> > Similarly, .bat files have inconsistent svn:eol-style properties:
> 
> Actually no, both should have CRLF line endings enforced. .bat files
> certainly should, at least once upon a time they would otherwise be invalid
> on Windows. For the .vcproj etc (XML) files, I'm not sure whether
> VisualStudio still cares, but the .properties, IIRC are treated differently?

So, shouldn't CRLF line endings be enforced for *all* .ezt and .bat files?

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

build/generator/templates/*.ezt and .bat files and svn:eol-style

[Vincent Lefevre]

A minor issue I had found during my checks for the URL update:

for i in build/generator/templates/*.ezt
do file $i; svn pg svn:eol-style $i; done

gives

build/generator/templates/build-outputs.mk.ezt: ASCII text, with very long 
lines (478)
native
build/generator/templates/build_locale.ezt: DOS batch file, ASCII text
native
build/generator/templates/pkg-config.in.ezt: ASCII text
native
build/generator/templates/svn_config.vcproj.ezt: exported SGML document, ASCII 
text
native
build/generator/templates/svn_config.vcxproj.ezt: exported SGML document, 
Unicode text, UTF-8 (with BOM) text
native
build/generator/templates/svn_locale.vcproj.ezt: exported SGML document, ASCII 
text
native
build/generator/templates/svn_locale.vcxproj.ezt: exported SGML document, ASCII 
text
native
build/generator/templates/vcnet_sln.ezt: ASCII text
native
build/generator/templates/vcnet_vcproj.ezt: exported SGML document, ASCII text
native
build/generator/templates/vcnet_vcxproj.ezt: exported SGML document, Unicode 
text, UTF-8 (with BOM) text
native
build/generator/templates/vcnet_vcxproj_filters.ezt: exported SGML document, 
ASCII text
native
build/generator/templates/vcnet_vsprops.ezt: exported SGML document, ASCII 
text, with CRLF line terminators
svn: warning: W200017: Property 'svn:eol-style' not found on 
'build/generator/templates/vcnet_vsprops.ezt'
svn: E20: A problem occurred; see other errors for details

Shouldn't build/generator/templates/vcnet_vsprops.ezt have
svn:eol-style set to native so that it doesn't get CRLF
line terminators under Unix?

Similarly, .bat files have inconsistent svn:eol-style properties:

for i in **/*.bat
do file $i; svn pg svn:eol-style $i; done

(under zsh or bash+globstar) gives

contrib/client-side/diff3wrap.bat: DOS batch file, ASCII text, with CRLF line 
terminators
CRLF
contrib/client-side/diffwrap.bat: DOS batch file, ASCII text, with CRLF line 
terminators
CRLF
subversion/tests/cmdline/svneditor.bat: DOS batch file, ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/config.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/do_all.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svnbuild.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svncheck.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svnclean.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svnlog.bat: ASCII text
native
tools/dev/benchmarks/suite1/run.bat: ASCII text
native

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [PATCH] URL update to https

[Vincent Lefevre]

On 2022-08-17 13:16:42 -0400, Nathan Hartman wrote:
> Looks good to me. Thanks for doing that. I'll commit it shortly.
> 
> Is there anything in particular you'd like the log message to say (other
> than credit for writing this patch of course)?

Nothing particular. Thanks.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [PATCH] URL update to https

[Vincent Lefevre]

On 2022-08-14 10:56:07 -0400, Nathan Hartman wrote:
> I'm attaching an updated patch with the following changes:
> 
> (1) Revert LICENSE
> (2) Revert the license headers

Your patch adds a trailing newline to 2 files:
  subversion/tests/cmdline/svneditor.bat
  tools/buildbot/slaves/win32-vcpkg/svntest-build.cmd

I've attached a new patch obtained with

  perl -pi -e \
's[http://(subversion\.apache\.org|svnbook\.red-bean\.com)]
  [https://\1];
 s[http://(www.apache.org)(?!/licenses)]
  [https://\1]' **/*(.)

and the manual formatting improvements on

  subversion/libsvn_repos/repos.c
  tools/dev/contribulyze.py

This corresponds to your patch without the added trailing newlines.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)


subversion-url-v3.patch.xz
Description: Binary data

Re: [BUG] svn patch with svn:eol-style set to native

[Vincent Lefevre]

On 2022-08-15 10:06:48 -0400, Nathan Hartman wrote:
> On Sun, Aug 14, 2022 at 8:08 PM Vincent Lefevre  
> wrote:
> > printf %s "$(printf "%d\n" `seq 2 8`)" > file
> 
> One thing I don't understand about the above is why does the outer
> printf %s strip the last newline from the inner one?

It is the $(...) that strips the trailing newline(s):

  
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_03

That's why I've used that.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[BUG] svn patch with svn:eol-style set to native

[Vincent Lefevre]

About this bug:

On 2022-08-14 14:11:15 -0400, Nathan Hartman wrote:
> But applying your original patch to a clean working copy and then
> running 'svn diff' does show it -- note the - and + of  and
> "\ No newline at end of file" at the end of the output:
[...]
> That looks like a bug in SVN. This machine (the one I'm using now) has
> 1.13.0 so I'll try this with 1.14.2 later...

I can reproduce the issue with svn 1.14.2, and this is due to
svn:eol-style being set to native.

To reproduce:


#!/bin/sh

set -ex

svn --version

mkdir my-test-svn
cd my-test-svn

svnadmin create svn
root=file://`pwd`/svn
svn co "$root" wc
cd wc

printf %s "$(printf "%d\n" `seq 8`)" > file
svn add file
svn ps svn:eol-style native file
svn ci -m 'add file'

printf %s "$(printf "%d\n" `seq 2 8`)" > file
svn diff > file.patch
cat file.patch
svn revert file
svn patch file.patch

svn diff

cd ../..
rm -rf my-test-svn


-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [PATCH] URL update to https

[Vincent Lefevre]

On 2022-08-14 10:56:07 -0400, Nathan Hartman wrote:
> Regarding LICENSE, I also think it should remain identical.

OK (if a change is needed, perhaps Apache could do it first, like
what the FSF did for GPLv3, in 2017 IIRC). However, I think that
there would be no problems to update the license location in
source files.

> Regarding URLs that became incorrect for other reasons, e.g.,
> https://www.apache.org/dev/new-committers-guide.html#cla, I think
> those should be addressed in separate patches...
[...]
This is also what I think.

> I'm attaching an updated patch with the following changes:
> 
> (1) Revert LICENSE
> (2) Revert the license headers
> (3) In a few files [1], no URLs were changed but a newline was added
> to the end of the file, which wasn't there before. I think this should
> be done in a separate patch.
> 
> Please check the updated patch...

I'll look at it later.

> [1] Those files are:
> build/generator/templates/svn_config.vcxproj.ezt
> build/generator/templates/svn_locale.vcxproj.ezt
> build/generator/templates/vcnet_vcxproj_filters.ezt
> subversion/libsvn_subr/win32_crashrpt_dll.h

Are you sure? I cannot see any such change in my patch.
Couldn't this be the result of the revert of the licence
headers?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[PATCH] URL update to https

[Vincent Lefevre]

Hi,

Following the thread "http URLs should be updated to https",
I've attached a patch that changes the http URLs to https for
the following hostnames:

  subversion.apache.org
  www.apache.org
  svnbook.red-bean.com

This has been done with

  perl -pi -e \
's[http://((subversion|www)\.apache\.org|svnbook\.red-bean\.com)]
  [https://\1]' **/*(.)

from the working copy (trunk), thus including the po files.

In addition to these changes, I have corrected the alignment of NL in
subversion/libsvn_repos/repos.c for the 2 concerned URLs and slightly
reformatted the text in tools/dev/contribulyze.py so that the
corresponding output fits again in 80 columns.

Note that the URLs of the LICENSE file are updated in this patch
while they haven't changed upstream. But since this doesn't change
the license itself, I suppose that this is correct.

I haven't changed the quoted http URL from tools/dev/check-license.py
manually ("http://www\.apache\.org/licenses/LICENSE-2\.0;, not matched
by the above regexp) because this appears to be obsolete.

A "make check" here gives

Summary of test results:
  2554 tests PASSED
  162 tests SKIPPED
  81 tests XFAILED (17 WORK-IN-PROGRESS)

In my checks, I've also found URL anchors that became incorrect in
the past (this is unrelated to this patch) and could be fixed in a
separate commit:

* In notes/api-errata/1.7/ra001.txt, change
  
https://subversion.apache.org/docs/api/latest/structsvn__delta__editor__t.html#_details
to
  
https://subversion.apache.org/docs/api/latest/structsvn__delta__editor__t.html#details

* In the following files
notes/commit-access-templates/full-committer.tmpl
notes/commit-access-templates/partial-committer.tmpl
notes/commit-access-templates/pmc-member.tmpl
the anchor in https://www.apache.org/dev/new-committers-guide.html#cla
no longer exists.

* In tools/examples/svnlook.py, the anchor in
https://subversion.apache.org/docs/community-guide/#docs
no longer exists.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)


subversion-url.patch.xz
Description: application/xz

[PATCH] autogen.sh errors from autoheader when python doesn't exist

[Vincent Lefevre]

When python doesn't exist (as opposed to python3), running autogen.sh
gives errors

/usr/bin/env: ‘python’: Permission denied

The autogen.sh script itself uses

PYTHON="`./build/find_python.sh`"

for the .py scripts it runs, but this is not sufficient for Python
scripts run by autoheader/autom4te, such as build/getversion.py.
A strace shows:

25execve("/bin/sh", ["sh", "-c", "$PYTHON build/getversion.py SVN 
subversion/include/svn_version.h"], 0x7ffe47752730 /* 125 vars */ 
[...]
26execve("build/getversion.py", ["build/getversion.py", "SVN", 
"subversion/include/svn_version.h"], 0x55aad83bcbb8 /* 125 vars */ 

while $PYTHON is empty and build/getversion.py starts with

#!/usr/bin/env python

Hence the issue. Adding "export PYTHON" in the autogen.sh script
as follows

-PYTHON="`./build/find_python.sh`"
+PYTHON="`./build/find_python.sh`"; export PYTHON

solves the issue. I've attached the corresponding patch.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Index: autogen.sh
===
--- autogen.sh	(revision 1903210)
+++ autogen.sh	(working copy)
@@ -170,7 +170,7 @@
 #   and we can state that dev people need Python on their machine. Note
 #   that running gen-make.py requires Python 2.7 or newer.
 
-PYTHON="`./build/find_python.sh`"
+PYTHON="`./build/find_python.sh`"; export PYTHON
 if test -z "$PYTHON"; then
   echo "Python 2.7 or later is required to run autogen.sh"
   echo "If you have a suitable Python installed, but not on the"

Re: missing --force option for "svn log" (useful with --diff)

[Vincent Lefevre]

On 2022-07-29 22:41:43 +0300, Pavel Lyalyakin via dev wrote:
> On Fri, Jul 29, 2022 at 5:30 PM Branko Čibej  wrote:
> >
> > On 16.07.2022 16:23, Vincent Lefevre wrote:
> > > When using "svn log --diff", one may need diff options.
> > > One has --diff-cmd, --internal-diff and -x, but --force
> > > is missing, though it would be useful too (in particular
> > > with --diff-cmd to textify diff inputs).
> >
> > What would --force do?
> 
> I believe that this post is relevant to Vincent's request:
> https://stackoverflow.com/q/72815562.

Yes, --force would behave just like with "svn diff" (similarly
to --diff-cmd, --internal-diff and -x, whice behave like with
"svn diff").

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

missing --force option for "svn log" (useful with --diff)

[Vincent Lefevre]

When using "svn log --diff", one may need diff options.
One has --diff-cmd, --internal-diff and -x, but --force
is missing, though it would be useful too (in particular
with --diff-cmd to textify diff inputs).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: http URLs should be updated to https

[Vincent Lefevre]

On 2022-05-08 17:51:30 +0200, Daniel Sahlberg wrote:
> Den fre 11 mars 2022 kl 12:10 skrev Vincent Lefevre  >:
> 
> > On 2022-03-11 10:29:12 +, Julian Foad wrote:
> > > Julian Foad wrote:
> > > > +1. Can you send a patch?
> > >
> > > By the way, the reason I ask if you would be willing, rather than "just
> > > quickly doing it" myself, is even a small "obvious" fix like this tends
> > > to require more than it initially looks like: checking if it's already
> > > done in head of trunk,
> >
> > This is what I'm looking at.
> >
> 
> Hi Vincent,
> 
> Did you get around to do anything about this? Even if you didn't complete
> it, maybe we can commit part of the job.
[...]

Hi Daniel,

Sorry for the late reply. I got busy for the past few months.
But I can now look at it.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: http URLs should be updated to https

[Vincent Lefevre]

On 2022-03-11 10:29:12 +, Julian Foad wrote:
> Julian Foad wrote:
> > +1. Can you send a patch?
> 
> By the way, the reason I ask if you would be willing, rather than "just
> quickly doing it" myself, is even a small "obvious" fix like this tends
> to require more than it initially looks like: checking if it's already
> done in head of trunk,

This is what I'm looking at.

> finding other similar places,

I typically do a recursive grep to find the potential updates to be
done.

> deciding if any of them shouldn't be updated for whatever reason,
> running the test suite, adjusting test results to match...

Yes, this requires some time to do it well (I've already done such
kinds of changes in another project), mainly for checking. I suppose
that eveything the end user can see needs to be updated, but also
for the readers of the source code, e.g. comments like

  #  See http://subversion.apache.org for more information.

I have a question for 2 kinds of files. I think that the URLs should
be updated too in these files, but I need confirmation:
  * The "CHANGES" file.
  * The .po files. The advantage is that this would avoid the need of
the update by the translators. A potential drawback is that the
additional character might need to adjust the formatting, e.g. to
fit on 80 columns where applicable (there aren't many concerned
lines, so that I could report any such formatting issue).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: A two-part vision for Subversion and large binary objects.

[Vincent Lefevre]

On 2022-03-11 10:04:36 +, Julian Foad wrote:
> Daniel Sahlberg wrote:
> > I'm taking an opposite position with regards on where this should be
> > administred. [...] I would prefer a multi-level approach where the
> > repository (through svn:foo properties) could suggest pristine-less WC
> 
> I understand completely your case, but the solution you need is a way to
> configure your client's behaviour remotely, and that is not necessarily
> best done by Subversion versioned properties. Do you see the
> distinction? Rather, what you need is for client configuration to be
> managed centrally and obeyed by your clients. The server and clients
> involved *could* be your Subversion repository server and Subversion
> clients, but could alternatively be some other mechanism. You just need
> some mechanism that works and is easy enough to deploy.

If I understand what Daniel Sahlberg intends to mean is that the
property would actually tell the client what to do *by default*,
removing the need to configure the client. But I suppose that its
use would be very uncommon (say, for a repository storing only
big binary files, the main goal being to keep the history, but
where "svn diff" would never be done in practice).

Having such a property on directories and/or individual files would
be much more interesting, but in such a case, there should be more
than 2 levels of suggestion.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

http URLs should be updated to https

[Vincent Lefevre]

When running "svn --help", I get:

[...]
For additional information, see http://subversion.apache.org/

This URL should be updated to https, and probably other instances
in the Subversion source.

http://svnbook.red-bean.com is also concerned.

Even though the websites automatically redirect to https, using
https URLs directly has at least 2 advantages:
  * this redirection is not always protected against man-in-the-middle
attacks;
  * using https would be a bit faster (by skipping the redirection).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: A two-part vision for Subversion and large binary objects.

[Vincent Lefevre]

On 2022-01-27 17:21:42 +, Julian Foad wrote:
> This setting doesn't have to be persistent in the WC. It could be
> configured in client run-time config instead (e.g.
> ~/.subversion/config), as we previously mentioned.
> 
> If it's stored in the WC then we need to create some new UI to control
> the setting. I am not sure we want to do so just now. It does seem, if
> we were designing svn from scratch, such a setting would ideally be
> remembered in the WC and there would be UI to control it, analogous to
> "git config --system|--global|--local", but we are not there.
> 
> When we were thinking the setting would be of the form "on for all files
> larger than X" then the runtime config seemed more appropriate, as that
> form might be expected to apply to many WCs, possibly adding conditions
> such as "and path to WC matches Y" or "repository matches Z". Specifying
> the WC path is ugly as WCs can be moved and we haven't ever exposed any
> other identifier for a WC. Specifying a pattern to match the WC path is
> better. Specifying it per repository is very logical because the
> behavior is so dependent on the repo connection.

But what if a WC can be accessed from different machines (e.g. via
NFS or SSHFS), so potentially with different ~/.subversion/config
files? And what if a WC is stored on a USB drive/disk, which can
move to various machines?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: A two-part vision for Subversion and large binary objects.

[Vincent Lefevre]

On 2022-01-21 11:15:04 +, Julian Foad wrote:
> Premature Hydrating
> 
> The present implementation "hydrates" (fetches missing pristines) every
> file within the whole subtree the operation targets. This is done by
> every major client operation calling svn_client__textbase_sync() before
> and afterwards.
> 
> That is pessimistic: the operation may not actually touch all these
> files if limited in any way such as by
> 
>   - depth filtering
>   - other filtering (changelist, properties-only, ...)
>   - terminating early (e.g. output piped to 'head')
> 
> That introduces all the fetching overhead for the given subtree as a
> latency before the operation shows its results, which for something
> small at the root of the tree such as "svn diff --depth=empty
> --properties-only ./" may make a significant usability impact.

Do you mean that "svn diff" at the root will fetch everything
even if no files are modified?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: A two-part vision for Subversion and large binary objects.

[Vincent Lefevre]

On 2022-01-13 22:38:34 -0600, Karl Fogel wrote:
> So if we have client-side configuration that can specify "no pristine" based
> on some combination of one or more of...
> 
>  - file size
>  - repository of origin
>  - path and/or basename
>  - svn:mime-type property (if present)
>  - some custom property

I would add: the age of the last change of a file in the repository.

My personal repository has many files (currently 26937), but most of
them are no longer changed.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Broken pipe on the diff command with --diff-cmd

[Vincent Lefevre]

On 2021-08-25 19:18:49 -0400, Nathan Hartman wrote:
> I don't see mention of it in the issue tracker. Please could you file
> it there?

Done: https://issues.apache.org/jira/browse/SVN-4879

> I started looking into it but ran out of time for now. Anyway, SIGPIPE
> is ignored throughout the client, for good reasons, since r848296:
> 
> (relevant bits of log message):
> 
>- Ignore SIGPIPE, to prevent wedging the repos on network errors,
> or when piping the output to a command that quits before all data
> is read, e.g. head, more or less.
> 
> SIGPIPE is disabled in the client (and 2 other programs) by:
> 
> +#ifdef SIGPIPE
> +  /* Disable SIGPIPE generation for the platforms that have it. */
> +  apr_signal(SIGPIPE, SIG_IGN);
> +#endif
> 
> It would be easy to temporarily re-enable it before executing diff-cmd
> but first it's important to make sure that won't break another use
> case which the current disabling is handling. (An example of one
> question that comes to mind: is it possible to use --diff-cmd with
> 'svn log -r --diff' and if so are there any ramifications to
> re-enabling SIGPIPE in that scenario? I haven't checked yet.)

I added a simpler suggestion in the report:

A possible fix (to be tested): instead of ignoring SIGPIPE (using
SIG_IGN), use a handler that does nothing. That way, when a command is
executed, SIGPIPE will be reset to the default action in the command
(i.e. killing the command, unless the command itself changes how it
handles SIGPIPE).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Broken pipe on the diff command with --diff-cmd

[Vincent Lefevre]

Hi,

I've found the following issue. When I pipe the output of
"svn diff --diff-cmd diff" and a broken pipe occurs on the
diff command, I get a "Broken pipe" error:

diff: standard output: Broken pipe
svn: E200012: 'diff' returned 2

Tested with svn, version 1.14.1 (r1886195) under Debian 11 (bullseye).

The cause is that svn runs the diff command with SIGPIPE ignored,
so that diff gets an EPIPE write error instead of being killed by
the signal. I think that a fix should be to reset SIGPIPE to the
default action just before executing the external diff command.

To reproduce the bug:


#!/bin/sh

set -e

export LC_ALL=C

mkdir my-test-svn
cd my-test-svn

svnadmin create svn
svn co file://`pwd`/svn wc
cd wc

seq 1 > file
svn add file
svn diff | head
svn diff --diff-cmd diff | head

cd ../..
rm -rf my-test-svn


This gives:

Checked out revision 0.
A file
Index: file
===
--- file(nonexistent)
+++ file(working copy)
@@ -0,0 +1,1 @@
+1
+2
+3
+4
+5
Index: file
===
--- file(nonexistent)
+++ file(working copy)
@@ -0,0 +1,1 @@
+1
+2
+3
+4
+5
diff: standard output: Broken pipe
svn: E200012: 'diff' returned 2

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: A two-part vision for Subversion and large binary objects.

[Vincent Lefevre]

On 2021-07-30 22:42:25 -0500, Karl Fogel wrote:
> On 30 Jul 2021, Daniel Shahaf wrote:
> > Increase the svndiff window size, so a single byte addition at the start
> > of the file doesn't result in $filesize/100KB delta ops?
> 
> Maybe?  I *think* that's a rare case, and if it is then it's probably not
> worth the implementation complexity.  I believe that when large blobby files
> get changed they tend to get changed all over, even when the semantic change
> is small (partly because their formats often have built-in compression or
> encryption).

This would still be useful in other contexts. I have big text files
for which the only changes are near the beginning, mainly because
of information stored in reverse-date order (e.g. changelogs).
These are not huge files ("only" several MBs), but still among the
biggest files of my repositories, and they can change quite often.
A textual diff command can handle them, so why not Subversion's
internal diff...

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2020-01-16 10:37:05 +, Julian Foad wrote:
> Branko Čibej wrote:
> > > Why are we overthinking this?
> 
> Indeed.  There is a lot of energy going in to this thread.  It's great to
> see energy and enthusiasm, but...
> 
> I know the current algorithm is sometimes annoying.
> 
> I accept sub-optimal decisions about it have been made in the past.
> 
> Any algorithm is going to be annoying in some cases.  If we were ever to
> change the algorithm, I suggest it would make sense to copy some well known
> one (e.g. from an implementation of unix 'ls' or a git command);

Note that ls and git may be bad examples since they are both local
(well, for ls, directories can be remote such as via NFS or SSHFS,
but I don't think that it has been written primarily for such cases).
For ls, I've checked that the output is correctly aligned, even
in the case of several thousands of files followed by a file with
user/group names larger than 8 characters.

For svn, it depends whether you can accept to have all the data
before outputting anything. For me, it would be OK, even with
the -R option.

> it would not make sense to invent another one of our own. But it
> does not makes sense to change the default algorithm again at all in
> svn 1.x series. It would be resonable only as part of a complete
> redesign of the command line interface some day.
> 
> Adding caching for this particular issue is really over-kill.  Caching is
> hard to get right.  (One issue, for example, is how revision properties can
> be modified on the server, and there is so far no mechanism for efficiently
> letting clients find out about the changes. That's one form of "stale cache"
> scenario to be figured out.)

This does not really apply in the particular case of "svn ls -v" since
only some length would be cached. The user will not get "old" data.
Just possibly columns larger than needed in case some long author name
no longer exists.

> On the other hand, if we invested in caching for more general use in the
> client, and then got username length information "for free" out of that,
> that would be a different matter.

Note that caching the logs would really be useful, as this is often
very slow to get some old logs.

[...]
> On the other hand, a more general configuration scheme to be able to specify
> the output line format for each of the main svn commands would be much more
> widely useful.  It would have advantages such as being able to generate
> abbreviated outputs tailored to certain use cases (e.g. in a private repo, I
> might not care to see my username at all); or emulating (to a limited
> extent) older versions of svn for compatibility with old scripts.

This would really be great. Not to have ISO 8601 dates is also
annoying.

> From a business point of view it would be hard to justify adding that
> flexibility -- it is hard to imagine many people making much use of it.

Most tools have much flexibility (this includes ls and git). I suppose
that people use them.

Various tools use a printf-like format (e.g. Mutt, zsh).

> But the more generic it is (e.g., available for all commands the
> print lines of output, not just 'ls' or just username fields) the
> more likely it is to be found useful. As this is open source, if
> someone wants to develop that, they can.
> 
> As I think out loud further down this road, I am thinking the fields used in
> the format template string should correspond exactly to the fields available
> in the XML output, using the same names.  In fact the line format output
> should be programmatically derivable from the XML.

I agree.

> And we have recently mentioned adding other formats such as JSON,
> which may be more widely useful in today's software tooling
> ecosystem.

Except that there are more tools to parse XML than JSON.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2020-01-13 16:51:25 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Mon, 13 Jan 2020 10:26 +00:00:
> > Good point. I was about to say that in most cases, these commands are
> > run from a working copy. But I now think that caching should be done
> > under the user's home (just like with most applications). Under Unix,
> > this would be under "$XDG_CACHE_HOME/subversion".
> 
> When would data be evicted from the cache?

In this particular case, I wonder whether there is any reason to
evict data from the cache. And the user can still entirely remove
the cache manually if need be.

> I'd say UUID, but we don't have to decide this now.

On 2020-01-14 13:19:08 +0100, Branko Čibej wrote:
> Why are we overthinking this? This information is really not relevant
> for integration with other tools:
> 
>   * GUIs don't need it; they can always dynamically resize columns in forms.

Dynamic resize is annoying: one is reading something, but this
suddenly goes away due to a resize.

>   * The command-line doesn't (really) need it; all information,
> including untruncated commiter names, is available in --xml mode.

Yes, but this needs to add a pipe or use a different command name
(this is no way to override the "svn ls" behavior... except by
using a wrapper, attempting to parse the arguments).

BTW, if untruncated committer names are available in --xml mode, it was
not necessary to change the behavior and break the column alignment.

> Adding a layer of caching is only going to add a stale-cache problem
> that we don't have now.

What stale-cache problem?

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2020-01-10 16:58:26 +0100, Johan Corveleyn wrote:
> Since both 'svn ls -v' and 'svn blame' can be called on a repository
> URL, I don't really see the point in caching the max(length(author))
> of the working copy.
> 
> The output from a URL would still be mis-aligned. It would make the
> presentation inconsistent between calling it on a working copy and on
> a URL.

Good point. I was about to say that in most cases, these commands are
run from a working copy. But I now think that caching should be done
under the user's home (just like with most applications). Under Unix,
this would be under "$XDG_CACHE_HOME/subversion".

The data should be associated with the URL of the repository.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2020-01-08 13:12:21 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Wed, Jan 08, 2020 at 10:28:01 +0100:
> > On 2020-01-07 16:10:52 +, Daniel Shahaf wrote:
> > > Vincent Lefevre wrote on Tue, Jan 07, 2020 at 16:17:10 +0100:
> > > > On 2019-12-23 06:35:08 +, Daniel Shahaf wrote:
> > > > > Two things are not immediately clear to me:
> > > > > 
> > > > > - This info is only needed by the cmdline client, not by other
> > > > >   library users. This suggests the API should be generic. A
> > > > >   per-client cache, maybe? There's already a
> > > > >   svn_client_ctx_t::client_name so it's not unprecedented.
> > > > 
> > > > Well, you don't know what the library users will do. Perhaps the cache
> > > > will benefit them too. The value can be used by them or not.
> > > 
> > > I think you've missed my point.  I'm saying we should try to design the 
> > > cache
> > > API in a way that will allow it to be used not only for 'svn ls -v' but 
> > > also
> > > for other things.  The API does not exist to serve the cmdline client; it
> > > exists to serve _all_ svn clients.
> > 
> > I entirely agree. But I'm still confused by your first sentence above
> > "This info is only needed by the cmdline client, not by other library
> > users." or perhaps you misplaced the "not"?
> 
> The length of the longest author name is needed by 'svn ls -v'.
> 
> The length of the longest author name is unlikely to be needed by 
> other consumers of the svn_client_* API.

A reimplementation of "svn ls -v" or "svn blame" may find this useful
information. Graphical clients may be interested in a cached full list
of authors to compute the size of the corresponding column.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2020-01-07 16:10:52 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Tue, Jan 07, 2020 at 16:17:10 +0100:
> > On 2019-12-23 06:35:08 +, Daniel Shahaf wrote:
> > > Two things are not immediately clear to me:
> > > 
> > > - This info is only needed by the cmdline client, not by other
> > >   library users. This suggests the API should be generic. A
> > >   per-client cache, maybe? There's already a
> > >   svn_client_ctx_t::client_name so it's not unprecedented.
> > 
> > Well, you don't know what the library users will do. Perhaps the cache
> > will benefit them too. The value can be used by them or not.
> 
> I think you've missed my point.  I'm saying we should try to design the cache
> API in a way that will allow it to be used not only for 'svn ls -v' but also
> for other things.  The API does not exist to serve the cmdline client; it
> exists to serve _all_ svn clients.

I entirely agree. But I'm still confused by your first sentence above
"This info is only needed by the cmdline client, not by other library
users." or perhaps you misplaced the "not"?

> > In any case, making the information available will not hurt.
> 
> It will hurt, actually. If we add the API we'll need to support it
> until 2.0 (= indefinitely), and everyone who ever tried to write
> code to the libsvn_client interface will have another function
> docstring to read and decide they don't care about.

Yes, perhaps this will have drawbacks at the API level, but the
information can be useful for various reasons, and the library
users could just decide to use it or not.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2019-12-23 06:35:08 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Mon, 23 Dec 2019 02:21 +00:00:
> > On 2019-12-21 08:09:46 +, Daniel Shahaf wrote:
> > > Vincent Lefevre wrote on Sat, Dec 21, 2019 at 00:09:09 +0100:
> > > > There's something wrong with "svn list -v" column alignment when
> > > > there are author names with more than 8 characters. For instance,
> > > > with the gcc repository:
> > > > 
> > > > [...]
> > > >  279442 jozeflDec 16 12:02 libgcc/
> > > >  278886 jvdelisle  Dec 01 23:29 libgfortran/
> > > > [...]
> > > 
> > > As implemented, the width is initially 8 but as soon as a longer name is 
> > > seen,
> > > subsequent lines will be aligned with that, up to a maximum length of 16.
> > > 
> > > Do you have a better algorithm to propose?
> > 
> > cache the maximum length name in the working-copy data.
> 
> Let's see.  We could cache the value in wc.db.  Running 'ls' or 'blame' [the
> same considerations apply to both] would update the cache; running 'cleanup'
> would purge it.
> 
> Two things are not immediately clear to me:
> 
> - This info is only needed by the cmdline client, not by other
>   library users. This suggests the API should be generic. A
>   per-client cache, maybe? There's already a
>   svn_client_ctx_t::client_name so it's not unprecedented.

Well, you don't know what the library users will do. Perhaps the cache
will benefit them too. The value can be used by them or not. In any
case, making the information available will not hurt.

> - What the cache keys should be. wc.db is designed to be shared
>   among multiple WC's, so presumably the cache should be per-wcroot
>   or possibly per-repository?

I don't understand. The wc.db file is inside the .svn directory of
the WC. How can it be shared by multiple WC's?

> Oh.  I missed it before, but passing -H increases the minimum width to 14.
> That's not obvious from the --help output and variable names.

But 14 is still a bit too small for me, and -H prevents the exact
sizes from being output.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2019-12-23 03:21:09 +0100, Vincent Lefevre wrote:
> On 2019-12-21 08:09:46 +, Daniel Shahaf wrote:
> > Vincent Lefevre wrote on Sat, Dec 21, 2019 at 00:09:09 +0100:
> > > There's something wrong with "svn list -v" column alignment when
> > > there are author names with more than 8 characters. For instance,
> > > with the gcc repository:
> > > 
> > > [...]
> > >  279442 jozeflDec 16 12:02 libgcc/
> > >  278886 jvdelisle  Dec 01 23:29 libgfortran/
> > > [...]
> > 
> > As implemented, the width is initially 8 but as soon as a longer name is 
> > seen,
> > subsequent lines will be aligned with that, up to a maximum length of 16.
> > 
> > Do you have a better algorithm to propose?
> 
> Make the minimum width configurable, or cache the maximum length name
> in the working-copy data.

Or wait for a fraction of second, say 0.1 s, after the first data
are available, and compute the width based on these initial data.
In most cases (e.g. when -R is not used), these data will actually
be complete, so that the output will be correct.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: "svn list -v" column alignment issue

[Vincent Lefevre]

On 2019-12-21 08:09:46 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Sat, Dec 21, 2019 at 00:09:09 +0100:
> > There's something wrong with "svn list -v" column alignment when
> > there are author names with more than 8 characters. For instance,
> > with the gcc repository:
> > 
> > [...]
> >  279442 jozeflDec 16 12:02 libgcc/
> >  278886 jvdelisle  Dec 01 23:29 libgfortran/
> > [...]
> 
> As implemented, the width is initially 8 but as soon as a longer name is seen,
> subsequent lines will be aligned with that, up to a maximum length of 16.
> 
> Do you have a better algorithm to propose?

Make the minimum width configurable, or cache the maximum length name
in the working-copy data.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

"svn list -v" column alignment issue

[Vincent Lefevre]

Hi,

There's something wrong with "svn list -v" column alignment when
there are author names with more than 8 characters. For instance,
with the gcc repository:

[...]
 279442 jozeflDec 16 12:02 libgcc/
 278886 jvdelisle  Dec 01 23:29 libgfortran/
[...]

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: truncated author names in 'svn ls -v' output

[Vincent Lefevre]

On 2018-12-07 15:10:39 +0100, Branko Čibej wrote:
> >> Did you have some specific change you wanted to propose?  (Other than
> >> making the 'author' column's width configurable)
> > I just wanted to have more than 8 characters for the authors names
> > (without changing the "file size" column).
> 
> Yes, the default width is still 8 columns but that will expand to up to
> 16 as needed, without any additional options.

OK, fine. Thanks for the information.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: truncated author names in 'svn ls -v' output

[Vincent Lefevre]

On 2018-12-07 12:06:08 +, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Fri, Dec 07, 2018 at 12:43:42 +0100:
> > the issue with the suffix solution (B/K/M/G) is that it is actually
> > less readable when one has many files in the list. I mean that with
> > the full file size displayed, it is very easy to see which files take
> > the most space, or similar information: the number of digits gives the
> > order of magnitude of the file size; this can be found in a fraction
> > of second. With the suffix form, one needs to look at all the sizes
> > one by one.
> 
> The output with SI suffixes is not the default.  If you don't opt-in to
> it by passing a new-in-1.12 option, you will never see it in the first
> place.

I'm a bit lost. :( I thought that this was the solution to make
more space for the author names (as the SI suffixes were discussed
under this thread).

> Did you have some specific change you wanted to propose?  (Other than
> making the 'author' column's width configurable)

I just wanted to have more than 8 characters for the authors names
(without changing the "file size" column).

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: truncated author names in 'svn ls -v' output

[Vincent Lefevre]

On 2018-11-26 18:19:59 +0100, Stefan Sperling wrote:
> On Mon, Nov 26, 2018 at 04:30:27PM +0100, Vincent Lefevre wrote:
> > On 2018-11-26 15:18:47 +0100, Branko Čibej wrote:
> > > Do please read the rest of the thread. A solution has already been
> > > implemented on trunk.
> > 
> > Except that this solution is not satisfactory for me. I guess that
> > the ultimate solution is to use --xml + a wrapper (thus one could
> > add things like coloring), though that's still limited as one
> > doesn't get the MIME type (svn:mime-type).
> 
> Please don't poke our grumpy Brane bear; we need him to stay in a good mood.
> You see, if you don't like Brane's solution, and you already have a
> working alternative of your own invention which you prefer anyway, then
> pointing out that Brane's solution is unsatisfactory to you just comes
> across as... kind of rude. Even if not meant that way, this was certainly
> not a constructive comment to make on a mailing list such as this one.

Well, if you want a constructive comment, the issue with the suffix
solution (B/K/M/G) is that it is actually less readable when one has
many files in the list. I mean that with the full file size displayed,
it is very easy to see which files take the most space, or similar
information: the number of digits gives the order of magnitude of the
file size; this can be found in a fraction of second. With the suffix
form, one needs to look at all the sizes one by one.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: truncated author names in 'svn ls -v' output

[Vincent Lefevre]

On 2018-11-26 15:18:47 +0100, Branko Čibej wrote:
> Do please read the rest of the thread. A solution has already been
> implemented on trunk.

Except that this solution is not satisfactory for me. I guess that
the ultimate solution is to use --xml + a wrapper (thus one could
add things like coloring), though that's still limited as one
doesn't get the MIME type (svn:mime-type).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: truncated author names in 'svn ls -v' output

[Vincent Lefevre]

On 2018-11-23 09:10:50 +0100, Stefan Sperling wrote:
> At 16 columns, name collisions become far less likely. In our own project,
> the output would now look like this:
[...]

Yes, but this makes less space for long filenames.

Why not make this size user-configurable, so that each user may choose
which size is best for him? (Ideally one should be able to choose a
configuration that depends on the repository.)

So...

> Index: subversion/svn/list-cmd.c
> ===
> --- subversion/svn/list-cmd.c (revision 1846851)
> +++ subversion/svn/list-cmd.c (working copy)
> @@ -150,7 +150,7 @@ print_dirent(void *baton,
>   dirent->size);
>  
>return svn_cmdline_printf
> -  (scratch_pool, "%7ld %-8.8s %c %10s %12s %s%s\n",
> +  (scratch_pool, "%7ld %-16.16s %c %10s %12s %s%s\n",
> dirent->created_rev,
> dirent->last_author ? dirent->last_author : " ? ",
> lock ? 'O' : ' ',

You could use * for the width and precision fields.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Checkpointing v1 design

[Vincent Lefevre]

On 2017-11-03 19:39:51 +0100, Branko Čibej wrote:
> On 03.11.2017 15:54, Julian Foad wrote:
> > After playing with two pre-prototypes and discussing a wide variety of
> > ideas on this list, I have given detailed thought to a v1
> > checkpointing design with the following properties:
> >
> >   * local (not server)
> >   * built on shelving v1 (patches)
> >   * main commands are 'checkpoint' to save a new version and
> > 'rollback' to revert to version N
> 
> The main issue I have with this is that "checkpoint" is not a verb. (I
> know, neither are 'changelist' or 'auth').
> 
> How about 'save' (to create a savepoint) and 'restore' instead?

Wouldn't it be better if these related commands have a common prefix?
A bit like property-related commands start with "prop".

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

On 2017-11-03 10:53:07 +0100, Vincent Lefevre wrote:
> Unfortunately, currently I can't. A few days ago, I tried to reproduce
> it with similar changes in a sample repository, but this didn't trigger
> the bug. I'll try again from information based on strace, i.e. on the
> fact that "perso" was moved in the past. But I fear that this may be
> more complex (otherwise the bug would have already been found).

As I thought, I couldn't reproduce the bug with such a sample
repository.

But I've found that the bug is fixed in the trunk. I could bisect
to search for the fix, and this is the following:


r1795116 | stefan2 | 2017-05-14 22:22:49 +0200 (Sun, 14 May 2017) | 17 lines

Fix issue #4677.

When non-exitent paths are allowed as a result of a DAG-walk
(as opposed to creating a "path not found" error on those),
we may encounter file nodes in the "parent" path.  Those are
not an error but simply an indication that there won't be any
sub-paths.

* subversion/libsvn_fs_fs/tree.c
  (open_path): If n/a paths are allowed, non-dir parents are, too.

* subversion/libsvn_fs_x/dag_cache.c
  (svn_fs_x__get_dag_path): Same.

* subversion/tests/libsvn_fs/fs-test.c
  (closest_copy_test_svn_4677): Add regression test.
  (test_funcs): Register new test.


https://issues.apache.org/jira/browse/SVN-4677 contains:

This bug was introduced in
http://svn.apache.org/viewvc?view=revision=1586953 , is
present in all releases since 1.9.0, and can be fixed by reverting
that commit.

And the test shows a similar error:

Updating '.':
svn: E160016: Failure opening '/b/fa/fb'
svn: E160016: '/b/fa' is not a directory in filesystem 
'1d2dc934-16b4-452c-a607-a6080764124e'

There is a comment saying:

"Backport for 1.9.6 is yet to be approved."

Could it really be backported?

-- 
Vincent Lefèvre <vinc...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

On 2017-11-03 10:01:26 +0100, Stefan Sperling wrote:
> I agree this looks like a bug. However, it is unclear how to reproduce it.
> 
> Can you provide a sample repository (or shell script) which triggers the bug?

Unfortunately, currently I can't. A few days ago, I tried to reproduce
it with similar changes in a sample repository, but this didn't trigger
the bug. I'll try again from information based on strace, i.e. on the
fact that "perso" was moved in the past. But I fear that this may be
more complex (otherwise the bug would have already been found).

However, I could do some tests if Subversion could log what it is
doing at the low level.

I suppose I could also try to find which commit triggered the bug.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

On 2017-11-03 00:18:59 +0100, Vincent Lefevre wrote:
> cventin:~> svn ls -r99809 
> file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
> svn: E160016: Failure opening '/perso/tcl/fidelite'
> svn: E160016: '/perso/tcl' is not a directory in filesystem 
> '99759db8-4ec0-0310-8bf9-df86780d22d8'

For this one, valgrind doesn't signal any issue. And with strace,
I can see that:

.../db/revs/99/99809 is opened 5 times.
.../db/revs/99/99807 is opened 3 times.
.../db/revs/52/52653 is opened 5 times.

r99807 is the revision where the /perso/tcl directory and the
/perso/tcl/fidelite file were added.

r52653 is the revision where the /misc/perso directory was renamed
to /perso.

-- 
Vincent Lefèvre <vinc...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

[Moving to dev]

On 2017-11-01 17:05:24 +0100, Vincent Lefevre wrote:
> Also, this is not consistent, unless future commits have an influence
> on the following command:
> 
> joooj:~> svn ls -r99809 
> file:///srv/d_joooj/home/svn/root/perso/tcl/fidelite@99808
> svn: E160016: Failure opening '/perso/tcl/fidelite'
> svn: E160016: '/perso/tcl' is not a directory in filesystem 
> '99759db8-4ec0-0310-8bf9-df86780d22d8'
> 
> ioooi:~> svn ls -r99809 
> file:///srv/d_ioooi/home/svn/root/perso/tcl/fidelite@99808
> fidelite
> 
> FYI, ioooi is the old server, which I migrated with dump+load to
> joooj at revision r102683. So, I should have obtained the same
> result; the one given by ioooi is the expected one.

This is definitely a bug, and a regression in 1.9.

I did the following test:

1. On a Debian/unstable machine (which has Subversion 1.9.7 r1800392),
I created a format 3 FSFS repository and loaded the dump.

2. I tested the issue:

cventin:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

cventin:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

cventin:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
svn: E160016: Failure opening '/perso/tcl/fidelite'
svn: E160016: '/perso/tcl' is not a directory in filesystem 
'99759db8-4ec0-0310-8bf9-df86780d22d8'

cventin:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
svn: E160016: Failure opening '/perso/tcl/fidelite'
svn: E160016: '/perso/tcl' is not a directory in filesystem 
'99759db8-4ec0-0310-8bf9-df86780d22d8'

So, this is buggy.

3. I copied the repository to a Debian/jessie machine (which has
Subversion 1.8.10 r1615264).

4. I tested the issue on this machine:

tomate:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

tomate:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

tomate:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

tomate:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

Everything is fine with the old version.

-- 
Vincent Lefèvre <vinc...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

[Vincent Lefevre]

On 2016-09-26 10:04:50 +0100, Julian Foad wrote:
> Daniel Shahaf wrote:
> > What would content hashes provide that comparing node-rev id's would not?
> 1. A node-rev id only exists for a tree that has been committed to
> the repository: there is no way to generate a node-rev id for an
> external tree of content client-side. Note what Paul Hammant wrote
> about the use case:
> 
> "I need to compare to a *local* representation of the same tree
> that's not under subversion control"

But what should the behavior be if there are keywords?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

svn does not react to Ctrl-C when a background process has an unredirected stdout

[Vincent Lefevre]

I've finally found a way to reproduce the following old bug:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=50

(it still occurs with svn 1.9.4).

Make SVN_SSH point to the following script:


#!/bin/sh

sleep 10 &

while true; do :; done


Do a "svn up" and Ctrl-C. Then svn terminates only when "sleep"
terminates. If I redirect the output of "sleep" to /dev/null,
like that:


#!/bin/sh

sleep 10 > /dev/null &

while true; do :; done


then the problem disappears, i.e. Ctrl-C immediately terminates svn.

Note: The reason this problem occurs in my case is that my script
does "ssh -fMN ..." (which corresponds to the "sleep ... &" here)
when the control socket doesn't exist. The infinite loop corresponds
to the real ssh that would freeze due to a network problem.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: random, unpredictable sort of svn diff

[Vincent Lefevre]

On 2016-05-08 17:56:00 +0200, Branko Čibej wrote:
> On 08.05.2016 15:55, Vincent Lefevre wrote:
> > I haven't found a bug on this subject on Jira.
> 
> This is hardly a bug, right.

I should have said issue (this includes improvements).

-- 
Vincent Lefèvre <vinc...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

random, unpredictable sort of svn diff

[Vincent Lefevre]

Hi,

The output of "svn diff" is still in random order (not even
pseudo-random, completely unpredictable). There had been discussions
in the past to fix this, e.g.

  
https://mail-archives.apache.org/mod_mbox/subversion-users/201203.mbox/%3C1332298973.3028.17.camel@segulix%3E

What's the status?

I haven't found a bug on this subject on Jira.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incomplete xml output when using --xml to non-existent server

[Vincent Lefevre]

On 2016-03-03 10:31:52 +0100, Johan Corveleyn wrote:
> No, of course not :-). I just gave an example where the output was
> broken (host not found), as opposed to another error condition (server
> reponds "URL 'X' non-existent in revision Y") where the xml response
> is still valid. Ignoring implementation (which I always do when I'm
> arguing about behavior), this seems quite weird to me.

No, this is different. In the former case, this is a server or
communication problem: it is not possible to output the info
because it is not possible to know what it is. In the latter case,
the communication is successful, so that it is possible to output
the info.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Unversioned files with invalid UTF-8 sequence in name confuse svn

[Vincent Lefevre]

On 2016-03-01 17:12:05 +0100, Branko Čibej wrote:
> On 01.03.2016 16:58, Markus Schaber wrote:
> > Hi, Bert,
> >
> > From: Bert Huijben [mailto:b...@qqmail.nl]
> >> From: Markus Schaber [mailto:m.scha...@codesys.com]
> >>> Hi, Brane and Vincent,
> >>>
> >>> From: Branko Čibej [mailto:br...@apache.org]
> > However Subversion doesn't handle that (BTW it would be much
> > better to
> > remember the expected locale by storing it in the .svn directory
> > 
> >
> > rather than giving obscure error messages: if it did, Subversion
> > would know that the user was using an incorrect locale without any
> > ambiguity).
>  And if the user changes the locale for valid reasons, the Subversion
>  working copy would break in a different way.
> >>> I guess we would need some "change locale" operation, which would at
> >>> least update the saved locale in the .svn directory.
> >> There is no saved locale in the .svn directory...
> > Currently, yes, but it was suggested in the discussion, see the line above.
> 
> The major problem with the "saved locale" idea is that it creates yet
> another potential discrepancy. I really can't imagine how we'd be doing
> the majority of our users a service by adding another knob that can
> seriously break things when it's misconfigured, but doesn't do anything
> useful most of the time.

I don't understand what you mean here. As you said, Subversion
expects the user to always use the same locale with a given working
copy.

Currently:

  * If the user always uses the same locale, then everything is fine
most of the time. But if some tool writes filenames that cannot
be interpreted in this locale (this was the beginning of this
thread), the working copy gets unusable until the user removes
these files manually (which can be tedious).

  * If the user changes his locale, then the working copy is in an
inconsistent state; it is either unusable, or usable but with
incorrect information, which could lead to incorrect commits.

If the locale were recorded in .svn:

  * If the user always uses the same locale, then everything would be
fine all the time. If some tool writes filenames that cannot be
interpreted in this locale, then svn would know that it is not
a problem due to the change of the locale, i.e. it would know that
the file is necessarily unversioned. So, "svn st" would no longer
have any reason to fail, and svn-clean could work as expected
without this risk of being wrong.

  * If the user changes his locale, then svn would be able to emit
a clear error message about the locale mismatch. Then the user
could easily know what was wrong and change back to the previous
locale (the one recorded in .svn).

So, this would be a big improvement.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Unversioned files with invalid UTF-8 sequence in name confuse svn

[Vincent Lefevre]

On 2016-02-29 17:00:01 +0100, Bert Huijben wrote:
> The problem is most likely not that they have an invalid utf-8 sequence in
> their name, but that your settings report that filenames are encoded in one
> way, while there is a file which name can't be expressed by that format.
> 
> You get this error when Subversion isn't able to convert the filename to its
> internal utf-8 format, which should be capable to express any valid
> filename. (If you declare that all filenames are utf-8, there wouldn't be a
> conversion, so in most cases not an error)
> 
> To just handle it as unversioned as you suggest we need to at least be able
> to express its name.

There are two ways to express a filename:
  1. The only from the OS (e.g., in POSIX, this is just a sequence
 of bytes).
  2. The one used by Subversion internally.

(2) is necessary for versioned files, but for unversioned files,
you do not need to do the (1) -> (2) conversion.

> As you found out cleanup is not going to help here... we just can't access
> this file (or directory, or symlink), so we can't delete it or anything to
> help you.

For "svn st", I do not try to access the file. A file with an invalid
name cannot be a versioned file anyway. So, it could also just be
ignored, and outputting a non-fatal warning would be sufficient, IMHO.
Note that even "svn st -q" fails!

Concerning svn-clean, I think that instead of failing, svn-clean
should fallback to some alternate way. After all, a part of its code
does not use the internal filename representation.

The problem is that it is too easy to create files with a name using
invalid UTF-8 sequences (in my case, it seems just to be due to a bug
in Automake or Libtool). But the user should not be required to find
them and delete manually.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Unversioned files with invalid UTF-8 sequence in name confuse svn

[Vincent Lefevre]

With:

svn, version 1.9.3 (r1718519)
   compiled Jan 16 2016, 04:46:46 on x86_64-pc-linux-gnu

I have a working copy where "make check" has created files whose
name contain invalid UTF-8 sequences. The consequence is that
such files confuse svn:

$ =svn st
svn: E22: Error converting entry in directory 
'/home/vlefevre/software/mpfr-3.1/tests' to UTF-8
svn: E22: Valid UTF-8 data
(hex: 04 10 40 04 04 04 02 01 46 04 40)
followed by invalid UTF-8 sequence
(hex: c0 2e 69 64)
zsh: exit 1 =svn st

I think that a fatal error is a bug, i.e. "svn st" should just report
that these files are unversioned. The requirement on the validity of
filenames should just apply to versioned files or files to be versioned.

These files can't even be removed with svn-clean, which reports the
same error.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Perl interface: SVN::Core::dirent_canonicalize segfaults on undef

[Vincent Lefevre]

I have reported the following Debian bug:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=812318

With Subversion 1.9.3:

$ perl -MSVN::Core -e 'SVN::Core::dirent_canonicalize(undef)'
zsh: segmentation fault (core dumped)  perl -MSVN::Core -e 
'SVN::Core::dirent_canonicalize(undef)'

In case of undef, I suppose that the Perl module should pass the empty
string to the library (for which there are no errors), or return some
other kind of error.

I have not tried other kinds of arguments, but in any case, the
module should make sure that a valid string is passed to the
library.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Perl module SVN::Client and undefined behavior

[Vincent Lefevre]

With the Perl module SVN::Client from Subversion 1.9.3,
if one does:


#!/usr/bin/env perl

use strict;
use SVN::Client;

my $svnc = SVN::Client->new;
$svnc->info('.', undef, undef, sub { }, 0);


one gets an assertion failure because the path is not canonical.
While assertion failures are acceptable in C, this is rather annoying
to get one in Perl instead of one of the proper error reporting
methods. I think that this should be regarded as a bug, and it's up
to the Perl module to make sure that the path is canonical before
calling the corresponding svn library function.

BTW, SVN::Core::dirent_canonicalize is not documented in the
SVN::Core(3perl) man page.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

dump-load of a repository modifies verbose log output: M line lost

[Vincent Lefevre]

On a Debian/unstable machine with Subversion 1.9.2 and the MPFR
repository:

I currently have:

zira:~> TZ=UTC svn log -r 1984 -v file:///home/vinc17/private/svn-mpfr

r1984 | vlefevre | 2002-07-23 16:22:08 + (Tue, 23 Jul 2002) | 2 lines
Changed paths:
   M /trunk/mul.c

Fixed permissions.



But after a dump-load:

zira:~> TZ=UTC svn log -r 1984 -v file:///home/vinc17/private/test-svn-mpfr

r1984 | vlefevre | 2002-07-23 16:22:08 + (Tue, 23 Jul 2002) | 2 lines

Fixed permissions.



The "M /trunk/mul.c" line has been lost!

FYI, this came from the CVS conversion, where the permissions of mul.c
were changed, but with no effect in the SVN repository.

I don't know whether this is due to a buggy change in the format of
the dump, or a bug in the load. The dump of this revision changed in
the following way for this revision:

 Node-path: trunk/mul.c
 Node-kind: file
 Node-action: change
-Text-delta: true
-Text-delta-base-md5: a79ede6a595f7bad6a4549c23ef9cd2b
-Text-delta-base-sha1: a9ce78ba643c727019f926d957f37414cfc1a050
-Text-content-length: 15
-Text-content-md5: a79ede6a595f7bad6a4549c23ef9cd2b
-Text-content-sha1: a9ce78ba643c727019f926d957f37414cfc1a050
-Content-length: 15

-[Contents]

This might be related to:

  http://subversion.tigris.org/issues/show_bug.cgi?id=4598

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: URL's of revisions on http://svn.apache.org/viewvc/

[Vincent Lefevre]

On 2015-08-19 23:36:15 +, Daniel Shahaf wrote:
 Stefan Sperling wrote on Wed, Aug 19, 2015 at 18:35:55 +0200:
  On Wed, Aug 19, 2015 at 05:36:16PM +0200, Vincent Lefevre wrote:
   Hi,
   
   It seems that in the past, URL's of the following form were used:
   
 http://svn.apache.org/viewvc?view=revisionrevision=r1686988
   
   This now gives an Invalid revision error / 404 Not Found. The URL's
   now have the following form:
   
 http://svn.apache.org/viewvc?view=revisionrevision=1686988
   
   This is quite annoying to break all the past links...
  
  Wrong list. Talk to ASF infra.
 
 (see https://www.apache.org/dev/infra-contact)
 
 Support for leading r in revnums was implemented upstream in issues
 #441 and #488.  The implementation was a one-line patch, which should be
 trivial to cherry-pick onto Infra's ViewVC installation if upgrading the
 viewvc installation to a release that includes those patches isn't an
 option.

Thanks. I've sent a mail to infrastructure.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

URL's of revisions on http://svn.apache.org/viewvc/

[Vincent Lefevre]

Hi,

It seems that in the past, URL's of the following form were used:

  http://svn.apache.org/viewvc?view=revisionrevision=r1686988

This now gives an Invalid revision error / 404 Not Found. The URL's
now have the following form:

  http://svn.apache.org/viewvc?view=revisionrevision=1686988

This is quite annoying to break all the past links...

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: keywords not updated after an update that doesn't change the file due to local changes

[Vincent Lefevre]

On 2015-06-23 11:04:24 +0200, Julian Foad wrote:
 On 23 June 2015 at 10:45, Julian Foad julianf...@btopenworld.com wrote:
  Vincent Lefevre wrote:
  I have the following problem under Debian with the subversion 1.8.13-1
  Debian package, but I don't think that this is Debian specific.
 
  It seems that svn update doesn't update keywords when the local
  modifications on the concerned file are the same as in the repository.
  [...]
 
  Thank you, Vincent.
 
 Please feel free to file an issue to track this if you wish.

Done here:
  http://subversion.tigris.org/issues/show_bug.cgi?id=4585

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

keywords not updated after an update that doesn't change the file due to local changes

[Vincent Lefevre]

I have the following problem under Debian with the subversion 1.8.13-1
Debian package, but I don't think that this is Debian specific.

It seems that svn update doesn't update keywords when the local
modifications on the concerned file are the same as in the repository.

The problem occurs on something like that:

1. Do 2 checkout's of the repository, where there's some file foo
   has an Id keyword. - wc1 and wc2.
2. In wc1, modify file foo.
3. In wc1, do svn commit.
4. In wc2, do the same modification.
5. In wc2, do svn update.

At (5), the local changes and the changes from the repository are
merged, and the file is unchanged. But the Id keyword should have
been updated.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Copyright year displayed by command-line tools

[Vincent Lefevre]

On 2015-03-20 03:58:24 -0500, Greg Stein wrote:
 Daniel Berlin stated many years ago that the years associated with
 copyright lines are meaningless.

There are lots of particular cases, depending on the country of
publication, the year of publication, and so on. According to

  https://copyright.cornell.edu/resources/publicdomain.cfm

(which lists all the rules for the U.S.), it appears that this was
needed in the past, but this is no longer the case for the U.S.
Other countries may have different rules. So, I think that it is
still better to make the copyright notice correct.

It may also be useful for the end user.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

commit-email.pl generates a wrong Date: header in some locales

[Vincent Lefevre]

Hi,

The commit-email.pl script generates a wrong Date: header in some
locales, e.g.

Date: Tue, 20 May 2014 04:21:21 PM +0200

AFAIK, the reason is that in the strftime line, it uses %X instead
of %T. Actually there may also be problems with %a and %b, and for
these ones, the only possibility seems to locally change the locale
to C (and possibly change it back after the strftime).

This problem is still present:

  
http://svn.apache.org/viewvc/subversion/trunk/contrib/hook-scripts/commit-email.pl.in?view=markup

(revision 1295006).

In old versions of the script, the Date: header was not included,
thus was generated by the mail server and was correct.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: E175013 svn diff failure (access forbidden) with 1.8.5 (regression)

[Vincent Lefevre]

On 2014-02-17 16:39:56 +, Philip Martin wrote:
 You might be able to use socat to debug the traffic.  Run a socat relay
 on the client machine, something like:
 
   socat -v TCP-LISTEN:9630,reuseaddr,fork OPENSSL:svn.apache.org:443,verify=0
 
 then run the client command, something like:
 
   svn ls http://localhost:9630/repos/asf/subversion
 
 and socat should dump the decoded SSL traffic.  You may have to use
 TCP6-LISTEN if the client cannot connect.  Disabling compression with
 
   --config-option servers:global:http-compression=no  
 
 makes it easier to see the body of the response.

I had to use TCP6-LISTEN. And it seems that compression was not used
by default. I've attached what I obtain after hiding potentially
private information.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
 2014/02/18 11:04:38.735902  length=518 from=0 to=517
OPTIONS /svn//subdir HTTP/1.1\r
Host: localhost:9630\r
User-Agent: SVN/1.8.5 (x86_64-pc-linux-gnu) serf/1.3.4\r
Content-Type: text/xml\r
Connection: keep-alive\r
Accept-Encoding: gzip\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
Content-Length: 131\r
\r
?xml version=1.0 encoding=utf-8?D:options 
xmlns:D=DAV:D:activity-collection-set/D:activity-collection-set/D:options
 2014/02/18 11:04:38.746249  length=278 from=0 to=277
HTTP/1.1 401 Authorization Required\r
Date: Tue, 18 Feb 2014 10:04:38 GMT\r
Server: Apache\r
WWW-Authenticate: Basic realm=Subversion repository \r
Content-Length: 401\r
Keep-Alive: timeout=15, max=50\r
Connection: Keep-Alive\r
Content-Type: text/html; charset=iso-8859-1\r
\r
 2014/02/18 11:04:38.746679  length=401 from=278 to=678
!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN
htmlhead
title401 Authorization Required/title
/headbody
h1Authorization Required/h1
pThis server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required./p
/body/html
 2014/02/18 11:04:38.747414  length=561 from=518 to=1078
OPTIONS /svn//subdir HTTP/1.1\r
Host: localhost:9630\r
Authorization: Basic \r
User-Agent: SVN/1.8.5 (x86_64-pc-linux-gnu) serf/1.3.4\r
Content-Type: text/xml\r
Connection: keep-alive\r
Accept-Encoding: gzip\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
Content-Length: 131\r
\r
?xml version=1.0 encoding=utf-8?D:options 
xmlns:D=DAV:D:activity-collection-set/D:activity-collection-set/D:options
 2014/02/18 11:04:38.757377  length=704 from=679 to=1382
HTTP/1.1 200 OK\r
Date: Tue, 18 Feb 2014 10:04:38 GMT\r
Server: Apache\r
DAV: 1,2\r
DAV: version-control,checkout,working-resource\r
DAV: merge,baseline,activity,version-controlled-collection\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/partial-replay\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://apache.org/dav/propset/fs/1\r
MS-Author-Via: DAV\r
Allow: 
OPTIONS,GET,HEAD,POST,DELETE,TRACE,PROPFIND,PROPPATCH,COPY,MOVE,LOCK,UNLOCK,CHECKOUT\r
Content-Length: 192\r
Keep-Alive: timeout=15, max=49\r
Connection: Keep-Alive\r
Content-Type: text/xml; charset=utf-8\r
\r
 2014/02/18 11:04:38.758354  length=192 from=1383 to=1574
?xml version=1.0 encoding=utf-8?
D:options-response xmlns:D=DAV:
D:activity-collection-setD:href/svn//!svn/act//D:href/D:activity-collection-set/D:options-response
 2014/02/18 11:04:38.796275  length=466 from=1079 to=1544
OPTIONS /svn//subdir HTTP/1.1\r
Host: localhost:9630\r
Authorization: Basic \r
User-Agent: SVN/1.8.5 (x86_64-pc-linux-gnu) serf/1.3.4\r
Accept-Encoding: gzip\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
Transfer-Encoding: chunked\r
\r
42\r
?xml version=1.0 encoding=utf-8?D:options xmlns:D=DAV: /\r
0\r
\r
 2014/02/18 11:04:38.805984  length=647 from=1575 to=2221
HTTP/1.1 200 OK\r
Date: Tue, 18 Feb 2014 10:04:38 GMT\r
Server: Apache\r
DAV: 1,2\r
DAV: version-control,checkout,working-resource\r
DAV: merge,baseline,activity,version-controlled-collection\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/partial-replay\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://apache.org/dav/propset/fs/1\r

Re: E175013 svn diff failure (access forbidden) with 1.8.5 (regression)

[Vincent Lefevre]

On 2014-02-14 18:34:52 +, Philip Martin wrote:
 Vincent Lefevre vincent-...@vinc17.net writes:
 
  With svn 1.8.5 under GNU/Linux (Debian unstable), I get an error
 
  svn: E175013: Access to '/svn/' forbidden
 
  when I do svn diff -r118:119 https://host//subdir;, but
  svn diff -r118:119 https://host//subdir/file; is OK.
 
  There's no such problem with svn 1.6.12 (r955767) on some other
  Debian machine and svn 1.7.9 (r1462340) on an Ubuntu machine.
[...]
 A network trace would help, but I've never had much success getting
 wireshark to decode an https conection even when I have access to the
 server.  It might be easier to use a debug version of the client to
 determine which request is failing.

Anyway since the error message is

  Access to '/svn/' forbidden

doesn't it mean that it tries to do an access to '/svn/'?
If it does, then that's the bug. Access to '/svn/' is really
forbidden and should not be needed (and it wasn't with the older
svn clients).

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: E175013 svn diff failure (access forbidden) with 1.8.5 (regression)

[Vincent Lefevre]

On 2014-02-14 18:34:52 +, Philip Martin wrote:
 Vincent Lefevre vincent-...@vinc17.net writes:
 
  With svn 1.8.5 under GNU/Linux (Debian unstable), I get an error
 
  svn: E175013: Access to '/svn/' forbidden
 
  when I do svn diff -r118:119 https://host//subdir;, but
  svn diff -r118:119 https://host//subdir/file; is OK.
 
  There's no such problem with svn 1.6.12 (r955767) on some other
  Debian machine and svn 1.7.9 (r1462340) on an Ubuntu machine.
 
  I can't find bugs with E175013 on the issue tracker.
 
  Note: I am not the admin of the svn server, so that I don't have
  information about it.
 
 Just to be clear, you mean that 
 
svn diff -r118:119 https://host//subdir
 
 returns a diff with 1.6.12 and 1.7.9 but with 1.8.5 it fails with
 E175013?

Yes, that's what I meant.

 A network trace would help, but I've never had much success getting
 wireshark to decode an https conection even when I have access to the
 server.  It might be easier to use a debug version of the client to
 determine which request is failing.

Couldn't svn write it in the error message?

 You may be able to determine which version of Subversion is in use on
 the server by examining the HTTP headers with something like curl -D-.

It doesn't say which version. Even for the WWW server, it just says:

Server: Apache

But if this is useful, I suppose I could get information from the
admin.

 What sort of change is r119?  Text/properties/adds/deletes/copies?

Just text change. And it is not specific to r119. All diffs between
two *repository* revisions seem to fail.

In a working copy, svn diff -r118:119 fails, but not svn diff -r118.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

E175013 svn diff failure (access forbidden) with 1.8.5 (regression)

[Vincent Lefevre]

With svn 1.8.5 under GNU/Linux (Debian unstable), I get an error

svn: E175013: Access to '/svn/' forbidden

when I do svn diff -r118:119 https://host//subdir;, but
svn diff -r118:119 https://host//subdir/file; is OK.

There's no such problem with svn 1.6.12 (r955767) on some other
Debian machine and svn 1.7.9 (r1462340) on an Ubuntu machine.

I can't find bugs with E175013 on the issue tracker.

Note: I am not the admin of the svn server, so that I don't have
information about it.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: https://www.vinc17.net/
100% accessible validated (X)HTML - Blog: https://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: AW: svn cleanup and unreferenced pristines

[Vincent Lefevre]

On 2014-01-13 08:27:34 +, Markus Schaber wrote:
  On 2014-01-13 03:51:08 +0100, Branko Čibej wrote:
   On 13.01.2014 03:43, Vincent Lefevre wrote:
I meant deltas like in the repository (but see below).
  
   When you say delta you have to also define against what. Otherwise
   it's just a not very efficient compression algorithm.
  
  Against another pristine. Then there are several ways to do that, just like
  BDB and FSFS used different methods for the repository.
  But any of them would be more memory efficient than no deltas at all (like
  currently).
 
 But the need to do delta decompression on each svn status will cost a lot
 of i/o and CPU.

Not if the base revisions are stored like now (so, it shouldn't be
exactly like FSFS, perhaps more like BDB or some mix, or something
like what distributed SCM's do...).

Some new test on another working copy (before and after clean-up):

xvii:~ du -ks wd/.svn/pristine wd.new/.svn/pristine
4434616 wd/.svn/pristine
468884  wd.new/.svn/pristine

And the corresponding repository size: 1477764

So, here, the total size of pristine copies is more than 9 times
the size taken after a cleanup, and 3 times the total size of the
repository.

 I'd welcome an option in SVN to cache the repository. (TortoiseSVN
 already has some caching for the log, IIRC).

Yes, caching the log would really be useful. I do a lot svn log.

 But then, the cache should not be per-wc, but in a system (or user)
 global directory, or we will end up with several copies of the same
 repository on the machine.

Yes.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: svn cleanup and unreferenced pristines

[Vincent Lefevre]

On 2014-01-10 14:11:08 -0500, C. Michael Pilato wrote:
 Just a reminder that there can be performance benefits to not being too
 aggressive in our pristine purging, since update-style operations will
 consult the pristine cache before slurping file contents from the
 server.

If the working copy is the on same machine as the server, this doesn't
make much sense to keep unreferenced pristines.

Moreover if you want to speed up things, keeping pristines may not
be the right way (in particular if the whole files are kept, not
deltas). Caching the repository or a part of the repository may be
better.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

filename encoding in the working copy and symbolic links

[Vincent Lefevre]

The fact that Subversion can choose different encodings, depending
on the context, for filenames in the working copy can break
symbolic links. But the Book seems to be silent on these encoding
issues with symbolic links. And I wonder whether there should be
a specific flag for each symbolic link saying whether recoding is
needed (wanted by the user) or not, as only the user knows how the
contents of a symbolic link should be interpreted.

FYI, a quote from http://austingroupbugs.net/view.php?id=251 (about
possible future limitations of filename encoding in POSIX):

  (0001725)
  dalias (reporter)
  2013-08-17 16:13

  [...]
  One further issue I just realized with any sort of escaping is
  symbolic links: what do you do with link content? If the escaping
  form is filesystem-type-specific (for instance, only used on
  network-mounted devices) and not used on the local system that
  forbids all unwanted characters, you would run into situations where
  the proper translation of the symbolic link contents depend on where
  it points to. This is of course unacceptable since symbolic link
  contents are pure strings and aren't necessarily even being used to
  access files they point to...

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Filename encoding in working copy (was: Check-out fails with LANG=C)

[Vincent Lefevre]

[Cc to the dev@ list]

On 2013-07-19 16:50:49 +0200, Stefan Sperling wrote:
 On Fri, Jul 19, 2013 at 04:32:50PM +0200, Vincent Lefevre wrote:
  [...]
  
  Actually I think that the encoding needs to be stored somewhere in the
  working copy. Otherwise even if the user never changes the encoding,
  problems may occur, and this is also true with the current behavior.
  Indeed it was said in the past that USB keys were supported. So, move
  a USB key to a different computer, where the encoding specified by the
  environment is different... and see what happens if you try to do an
  svn update...
 
 Simply storing the encoding doesn't really solve anything. Sure, it
 remembers the LC_CTYPE value as the time the working copy was created.
 But then... what?

At least it would work better. And in some cases, one wouldn't notice
any problem. For instance, a repository (and therefore the corresponding
working copies too) may contain mostly filenames with only US-ASCII
characters, and when moving a USB key to another computer, one may be
interested only in a part of the working copy with such filenames.

 We also need to specify some new behaviour that increases user
 convenience for such a new feature to have any value.
 
 For this, we need answers to questions like:
 How can the client detect whether the stored encoding name matches
 the on-disk encoding? What does it do when they differ? How can users
 re-encode filenames in the working copy when on-disk encoding has changed? 

I don't understand what you mean here. If the chosen encoding for the
filenames is stored in the working copy: The on-disk encoding has been
chosen by Subversion (at checkout time). So, initially it matches the
stored encoding. This normally doesn't change... unless the user has
explicitly chosen to re-encode the filenames globally with some tool.
In such a case, the user also needs to modify the stored encoding, and
Subversion can provide a command for that. Subversion can also provide
a command to re-encode the filenames of a working copy and update the
stored encoding.

The main problem is when a working copy is moved to another machine,
on which the chosen encoding is not supported on the new machine. But
I don't think Subversion can do anything about it. The user needs to
make sure that the chosen encoding is supported on the new machine.
This should be the case for ASCII (+ escaping mechanism) and UTF-8.

 I'm very much interested in enhancements in this area, but at this
 point we don't need to rehash all the problems there are. We need
 to design solutions. This discussion needs a change of direction towards
 being more constructive, or it will die with no results. The discussion
 is also increasingly off-topic for the users@ list.
 
 In other words, I'm happy to continue this discussion on the dev@ list
 and review your proposed design specs and patches there.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Memory leak in svn log (1.7.5) with svn+ssh?

[Vincent Lefevre]

On Debian and Ubuntu GNU/Linux machines, when I run svn log in some
working copy or on some repository URL using the svn+ssh scheme, such
as with

  svn log | tail

or, if this is too fast,

  svn log | less

(and searching for a pattern near the end of the output), I notice
that the RES memory of the svn process increases, e.g. from 5MB to
24MB, while I would expect the needed memory to remain constant, or
at least bounded.

No such problem with the file scheme: the RES memory remains
constant.

$ svn --version
svn, version 1.7.5 (r1336830)
   compiled Sep 28 2012, 11:22:04

Copyright (C) 2012 The Apache Software Foundation.
This software consists of contributions made by many people; see the NOTICE
file for more information.
Subversion is open source software, see http://subversion.apache.org/

The following repository access (RA) modules are available:

* ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
  - handles 'http' scheme
  - handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
  - with Cyrus SASL authentication
  - handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
  - handles 'http' scheme
  - handles 'https' scheme

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

On 2013-03-05 16:52:30 +, Julian Foad wrote:
 Vincent Lefevre wrote:
[about server-side vs client-side]
  But even if there would be no problems with the
  construction/reconstruction, it would be a bad solution, IMHO.
  Indeed, for a commit, it is the client that is supposed to expand
  the data before sending the diff to the server,
 
 What do you mean the client [...] is supposed to expand the data? 
 I don't understand why you think the client is supposed to do such
 a thing.

Because the diff between two huge compressed files is generally huge
(unless some rsync-friendly option has been applied, when available).
So, if the client doesn't uncompress the data for the server, it will
have to send a huge diff or a huge compressed file, even though the
diff between the uncompressed data may be small. So, if
deconstruction/reconstruction is possible (canonical form),
it is much more efficient to do this on the client side.

   That point _is_ specific to a server-side solution.  With a
   client-side solution, the user's word processor may not mind if a
   versioning operation such as a commit (through a decompressing
   plug-in) followed by checkout (through a re-compressing plug-in)
   changes the bit pattern of the compressed file, so long as the
   uncompressed content that it represents is unchanged.
  
  I disagree.
 
 It's not clear what you disagree with.

With the second sentence (... may not mind ...), thus with the first
sentence too.

  The word processor may not mind (in theory, because
  in practice, one may have bugs that depend on the bit pattern,
  and it would be bad to expose the user to such kind of bugs and
  non-deterministic behavior), but for the user this may be important.
  For instance, a different bit pattern will break a possible signature
  on the compressed file.
 
 I agree that it *may* be important for the user, but the users have
 control so they can use this client-side scheme in scenarios where
 it works for them and not use it in other scenarios.

But one should need a scheme that will also work in the case where
users care about the bit pattern of the compressed file.

Moreover even when the users know that the exact bit pattern of the
compressed file is not important at some time, this may no longer
be true in the future. For instance, some current word processor may
ignore the dates in zip files, but future ones may take them into
account. So, you need to wonder what data are important in a zip
file, including undocumented ones used by some implementations (as
the zip format allows extensions). Taking them into account when it
appears that these data become meaningful is too late, because such
data would have already been lost in past versions of the Subversion
repository.

On 2013-03-05 17:10:02 +, Julian Foad wrote:
 I (Julian Foad) wrote:
  Vincent Lefevre wrote:
   On 2013-03-05 13:30:28 +, Julian Foad wrote:
  Vincent Lefevre wrote:
  On 2013-03-01 14:58:10 +, Philip Martin wrote:
  A server-side solution is difficult.  Suppose the client has some
  uncompressed content U which it compresses to C and sends to the server.
  The server can uncompress C to get U but unless the compression scheme
  has a canonical compressed form, with no other forms allowed, the server
  cannot avoid storing C because there is no guarantee that C can be
  reconstructed from U.
  
  This is not specific to server side. Even on the client side, the
  reconstruction may not be always possible, e.g. if the system is
  upgraded or if NFS is used. And the compression level may need to
  be detected or provided in some way.
  
  Hi Vincent.  I'm not sure you understood Philip's point.
  
  This should be more clear about what I meant below. What I'm saying is
  that whether this is done entirely on the server side (a bad solution,
  IMHO) or on the client side (see below why), the problems are similar.
  
  The point Philip made is *not* a problem if done client-side;
 
 Let me take that back.  The point that I interpreted as being the
 most significant impact of what Philip said, namely that the
 Subversion protocols and system design require reproducible content,
 is only a problem when done server-side.  Other impacts of that same
 point, such as you mentioned, are applicable no matter whether
 server-side or client-side.

The Subversion protocols and system design *currently* require
reproducible content, but if new features are added, e.g. due to the
fact that the users don't mind about the exact compressed content of
some file, then it could be decided to change the protocols and the
requirements (the server could consider some canonical uncompressed
form as a reference).

[...]
 So my main point is that the server-side expand/compress is a
 non-starter of an idea, because it violates basic Subversion
 requirements, whereas client-side is a viable option for some use
 cases.

I would reject the server-side expand/compress, not because of the
current requirements

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

On 2013-03-06 18:55:55 +, Julian Foad wrote:
 I don't know if anything like that would be feasible.  It may be
 possible in theory but too complex in practice.  The parameters we
 need to extract would include such things as the Huffman coding
 tables used and also parameters that influence deeper implementation
 details of the compression algorithm.  And of course for each
 compression algorithm we'd need an implementation that accepts all
 of these parameters -- an off-the-shelf compression library probably
 wouldn't be support this.

The parameters could also be provided by the user, e.g. via a svn
property. For instance, if the user wants some file file.xz to
be handled uncompressed, he can add a svn:compress property whose
value is xz -9 (if the -9 option was used). Then the client
would do a unxz on the file. If the user wants the bit pattern
to be preserved, the client would also do a xz -9 on the unxz
output. If some command fails or the result is not identical to
the initial file (for preserved bit pattern), the file would be
handled compressed (or the client should issue an error message
if requested by the user). Otherwise the file could be handled
uncompressed. This is the basic idea. Then there are various
implementation choices, such as whether the commands should be
part of Subversion or external commands provided by the system.

With a property, it would not be possible to change the behavior
on past revisions, but tools could do that on a svn dump.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

On 2013-03-01 14:58:10 +, Philip Martin wrote:
 A server-side solution is difficult.  Suppose the client has some
 uncompressed content U which it compresses to C and sends to the server.
 The server can uncompress C to get U but unless the compression scheme
 has a canonical compressed form, with no other forms allowed, the server
 cannot avoid storing C because there is no guarantee that C can be
 reconstructed from U.

This is not specific to server side. Even on the client side, the
reconstruction may not be always possible, e.g. if the system is
upgraded or if NFS is used. And the compression level may need to
be detected or provided in some way.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

Hi Julian,

On 2013-03-05 13:30:28 +, Julian Foad wrote:
 Vincent Lefevre wrote:
 
  On 2013-03-01 14:58:10 +, Philip Martin wrote:
   A server-side solution is difficult.  Suppose the client has some
   uncompressed content U which it compresses to C and sends to the server.
   The server can uncompress C to get U but unless the compression scheme
   has a canonical compressed form, with no other forms allowed, the server
   cannot avoid storing C because there is no guarantee that C can be
   reconstructed from U.
  
  This is not specific to server side. Even on the client side, the
  reconstruction may not be always possible, e.g. if the system is
  upgraded or if NFS is used. And the compression level may need to
  be detected or provided in some way.
 
 Hi Vincent.  I'm not sure you understood Philip's point.

This should be more clear about what I meant below. What I'm saying is
that whether this is done entirely on the server side (a bad solution,
IMHO) or on the client side (see below why), the problems are similar.

 His point is (correct me if I'm wrong) that Subversion's design
 requires that during a checkout or update, the server must
 reconstruct a file containing exactly the same bit pattern that the
 client sent when committing the file.  Compression schemes in
 general don't guarantee that expanding and then compressing will
 produce the same compressed bit pattern, even if you take care to
 use the same compression level.  Therefore, the server cannot
 simply expand the data before storing it and then re-compress it
 during checkout or update, because, although the resulting
 compressed file would be a valid representation of the user's data,
 it would not satisfy Subversion's own requirement that the bit
 pattern be identical to what was sent by the client during the
 commit.

You say that the server expands the data before storing it. This is
for a server-side only solution, I assume. But even if there would
be no problems with the construction/reconstruction, it would be a
bad solution, IMHO. Indeed, for a commit, it is the client that is
supposed to expand the data before sending the diff to the server,
and for an update, it is the client that is supposed to recompress
the data before storing it to the WC. Actually, the server doesn't
need to know how the file was compressed, it just needs to record
information about the compression (but doesn't need to know what
this means exactly).

 That point _is_ specific to a server-side solution.  With a
 client-side solution, the user's word processor may not mind if a
 versioning operation such as a commit (through a decompressing
 plug-in) followed by checkout (through a re-compressing plug-in)
 changes the bit pattern of the compressed file, so long as the
 uncompressed content that it represents is unchanged.

I disagree. The word processor may not mind (in theory, because
in practice, one may have bugs that depend on the bit pattern,
and it would be bad to expose the user to such kind of bugs and
non-deterministic behavior), but for the user this may be important.
For instance, a different bit pattern will break a possible signature
on the compressed file.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

On 2013-02-28 08:37:51 -0800, Ben Reser wrote:
 3) You'd be saving storage at the expense of using time (read: CPU) on
 every client that's working with those files when checking out.  So
 the end result may be worse than the current problem.

But storage is permanent, while a checkout may occur very rarely.
Of course, the decompress/recompress solution should be a user
choice because this may be bad in some cases, but it can also be
very useful.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

On 2013-02-28 10:58:07 -0800, Ben Reser wrote:
 Speaking with Julian here at ApacheCon he mentioned that gzip has a
 rsyncable option.  Looking into this turns out that there is a patch
 applied to Debian's gzip that provides this option.

I can't see such an option in Debian's gzip (from unstable, and
there is no experimental version).

 It resets the compression algorithm every 1000 bytes and thus makes
 blocks that can be saved between revisions of the file.

Wouldn't this work only when data are appended to the file?
If data are inserted or deleted, this would change the block
boundaries. Instead of fixed-length blocks, I'd rather see
boundaries based on the file contents.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: FSFS format7 and compressed XML bundles

[Vincent Lefevre]

On 2013-03-01 14:24:07 +, Philip Martin wrote:
 $ gzip --help | grep rsync
   --rsyncable   Make rsync-friendly archive
 $ dpkg -s gzip | grep Version
 Version: 1.5-1.1

OK, then it seems that its man page is out-of-date.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [RFC] Deprecate Berkelety DB filesystem backend

[Vincent Lefevre]

On 2013-01-05 10:35:52 +, Philip Martin wrote:
 Branko Čibej br...@wandisco.com writes:
 
* The BDB backend is an order of magnitude slower on trunk than FSFS
o timing parallel make check on my 4x4-core i7+ssd mac:
+ FSFS: real 7m33.213s, user 19m8.075s, sys 10m54.739s
+ BDB: real 35m17.766s, user 15m28.395s, sys 11m58.824s
 
 I don't see that behaviour.  On my machine with 1x2 Core2 Duo, Linux and
 SSD:
 
 FSFS: real 12m42.383s, user 7m37.557s, sys 5m58.534s
  BDB: real 11m30.895s, user 8m14.603s, sys 5m45.358s
 
 I don't think either your results or mine indicate anything about the
 relative performance of the backends as all the repositories in the
 testsuite are far too small.

As a user I no longer mind about BDB. But the above timings make
me think that BDB could still be useful for testing purpose (in
particular because you have no other backends).

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Bigger --deltas dump with 1.7.5 than with 1.6.17

[Vincent Lefevre]

On 2012-06-20 02:48:23 +0200, Vincent Lefevre wrote:
 On 2012-06-19 19:41:51 +0300, Daniel Shahaf wrote:
  I assume that the binary svndiff chunks are different, right?
 
 Yes, the binary svndiff chunks are different and have the declared
 size. But why is 1.6.17 better than 1.7.5?

In fact, after testing on another reprository, the difference doesn't
come from the client version, but from which version has created the
repository. Example:

$ svnadmin dump ~svn/root.bak --incremental --deltas -r0:16  dump1
$ svnadmin create tmprep
$ svnadmin load --force-uuid tmprep  dump1
$ svnadmin dump tmprep --incremental --deltas -r0:16  dump2
$ ls -l dump?
-rw-r--r-- 1 vinc17 vinc17 1118461 2013-01-23 15:37:58 dump1
-rw-r--r-- 1 vinc17 vinc17 1118490 2013-01-23 15:38:31 dump2
$ diff -a -u dump?
--- dump1   2013-01-23 15:37:58.0 +0100
+++ dump2   2013-01-23 15:38:31.0 +0100
@@ -37613,12 +37613,12 @@
 Text-delta: true
 Text-delta-base-md5: c2ce1c228d464a37bf8c440eb1dff95a
 Text-delta-base-sha1: 3fa18fdd2b987c96c5a62c97b97be89db2c58fe9
-Text-content-length: 11066
+Text-content-length: 11095
 Text-content-md5: 6c17738e070af8b69a4814ddc7484f69
 Text-content-sha1: ccbb2233b14ee51093a0c0bd08bc26ba1e88ee46
-Content-length: 11066
+Content-length: 11095
[...]

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Working copy corruption with svn 1.6.17 (r1128011) under Ubuntu

[Vincent Lefevre]

A few weeks ago, I got a working copy corruption (Checksum mismatch)
with svn 1.6.17 (r1128011) under Ubuntu. I tried to reproduce it with
similar commands on a test repository, without success. I don't know
whether this is a known bug that has been fixed, but just in case,
I give the list of commands I did below (I suspect that it is related
to the svn revert, but I'm not sure).

ioooi:~/config popd
~/config/etc/logrotate.d ~
ioooi:...g/etc/logrotate.d emacs apache2
ioooi:...g/etc/logrotate.d scd
Index: apache2
===
--- apache2 (revision 54297)
+++ apache2 (working copy)
@@ -24,7 +24,7 @@
compress
delaycompress
notifempty
-   create 660 vinc17 www-vinc17
+   create 664 vinc17 www-vinc17
sharedscripts
postrotate
/etc/init.d/apache2 reload  /dev/null
ioooi:...g/etc/logrotate.d svn ci
Log message has been cleaned up
Sendinglogrotate.d/apache2
Transmitting file data .
Committed revision 54298.
ioooi:...g/etc/logrotate.d popd
~
ioooi:~ cd config
ioooi:~/config cd etc
ioooi:~/config/etc cd ~/config.old/etc/awstats
ioooi:...g.old/etc/awstats \cp /srv/GANDIOS/etc/awstats/* .
ioooi:...g.old/etc/awstats svn add awstats-apache.conf
A awstats-apache.conf
ioooi:...g.old/etc/awstats svn ps svn:mime-type text/plain awstats-apache.conf
property 'svn:mime-type' set on 'awstats-apache.conf'
ioooi:...g.old/etc/awstats svn ps source 
ioooi:/etc/awstats/awstats-apache.conf awstats-apache.conf
property 'source' set on 'awstats-apache.conf'
ioooi:...g.old/etc/awstats ..
ioooi:~/config.old/etc svn ci
Log message has been cleaned up
Adding etc/awstats/awstats-apache.conf
Transmitting file data .
Committed revision 54299.
ioooi:~/config.old/etc ll awstats
total 16
-rw--- 1 vinc17 vinc17  384 2012-08-18 14:33:22 awstats-apache.conf
-rw-r--r-- 1 vinc17 vinc17  240 2012-08-18 14:32:30 awstats.conf.local
-rw-r--r-- 1 vinc17 vinc17 5982 2012-08-18 14:32:30 awstats.www.vinc17.net.conf
ioooi:~/config.old/etc ..
ioooi:~/config.old svn up
At revision 54299.
ioooi:~/config.old cd ../config.old/etc/awstats
ioooi:...g.old/etc/awstats ll
total 16
-rw--- 1 vinc17 vinc17  384 2012-08-18 14:33:22 awstats-apache.conf
-rw-r--r-- 1 vinc17 vinc17  240 2012-08-18 14:32:30 awstats.conf.local
-rw-r--r-- 1 vinc17 vinc17 5982 2012-08-18 14:32:30 awstats.www.vinc17.net.conf
ioooi:...g.old/etc/awstats popd
~/config.old ~/config.old/etc ~/config/etc ~/config ~
ioooi:~/config.old cd ../config/etc
ioooi:~/config/etc svn mkdir awstats
A awstats
ioooi:~/config/etc svn cp 
../../config.old/etc/awstats/awstats.www.vinc17.net.conf .
A awstats.www.vinc17.net.conf
ioooi:~/config/etc svn revert awstats.www.vinc17.net.conf
Reverted 'awstats.www.vinc17.net.conf'
ioooi:~/config/etc svn st
?   awstats.www.vinc17.net.conf
A   awstats
ioooi:~/config/etc rm awstats.www.vinc17.net.conf
ioooi:~/config/etc svn cp 
../../config.old/etc/awstats/awstats.www.vinc17.net.conf awstats/
A awstats/awstats.www.vinc17.net.conf
ioooi:~/config/etc cd ./awstats
ioooi:~/config/etc/awstats emacs awstats.www.vinc17.net.conf
ioooi:~/config/etc/awstats ..
ioooi:~/config/etc svn st
A   awstats
A  +awstats/awstats.www.vinc17.net.conf
ioooi:~/config/etc svn ci
Log message has been cleaned up
Adding etc/awstats
Adding etc/awstats/awstats.www.vinc17.net.conf
Transmitting file data .
Committed revision 54300.
ioooi:~/config/etc ..
ioooi:~/config svn up
svn: Checksum mismatch for 'etc/logrotate.d/apache2'; expected: 
'c628f4befcafbd72c75667605a93cdbe', recorded: '7753b31866179f6c4576b4f9f0ea865a'
ioooi:~/config svn up
   C etc
At revision 54300.
Summary of conflicts:
  Tree conflicts: 1
ioooi:~/config svn up
At revision 54300.
ioooi:~/config svn st
! C etc
 local add, incoming add upon update
!   etc/logrotate.d
ioooi:~/config svn --version
svn, version 1.6.17 (r1128011)
   compiled Dec 17 2011, 16:12:52
[...]

ioooi:~/config svn info ~/config
Path: /srv/d_ioooi/home/vinc17/config
URL: svn+ssh://mysvn/config/host/gandi.ioooi
Repository Root: svn+ssh://mysvn
Repository UUID: 99759db8-4ec0-0310-8bf9-df86780d22d8
Revision: 54300
Node Kind: directory
Schedule: normal
Last Changed Author: vinc17/ioooi
Last Changed Rev: 54300
Last Changed Date: 2012-08-18 15:11:56 +0200 (Sat, 18 Aug 2012)

ioooi:~/config svn info ~/config.old
Path: /srv/d_ioooi/home/vinc17/config.old
URL: svn+ssh://mysvn/config/host/old.gandi.ioooi
Repository Root: svn+ssh://mysvn
Repository UUID: 99759db8-4ec0-0310-8bf9-df86780d22d8
Revision: 54299
Node Kind: directory
Schedule: normal
Last Changed Author: vinc17/ioooi
Last Changed Rev: 54299
Last Changed Date: 2012-08-18 14:36:07 +0200 (Sat, 18 Aug 2012)

Note:
7753b31866179f6c4576b4f9f0ea865a is the final md5sum of the text-base.
c628f4befcafbd72c75667605a93cdbe is the md5sum of the text-base r54295.

At r54296, the file was modified and the svn:keywords 

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-03 23:31:55 +0200, Johan Corveleyn wrote:
 So I believe that 1.6's behavior in the working copy is indeed
 incorrect, and that the LCR of a child was always intended to be
 unaffected by the move/copy of a parent dir. That's the behavior of
 the repository, in both 1.6 and 1.7. What you saw with 1.6 (LCR 3 on
 file.txt) was only true in the originating working copy, which means
 that all other working copies will get LCR 2. Not good :-). And that
 has apparently been fixed in 1.7.

Actually, in the past, this was not regarded as incorrect. There has
already been a fix in 1.4 and 1.5 the other way around concerning
this particular problem (in addition to other problems). See

  http://subversion.tigris.org/issues/show_bug.cgi?id=1743

and the latest paragraph of my bug report (and the duplicates).

The problem now is that the URL + the LastChangedRev together can
no longer identify the file, because the LastChangedRev can now be
incorrect as a peg revision.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-04 12:51:13 +0100, Philip Martin wrote:
 If you are saying that a move should update the LastChangedRev of every
 file (and dir?) in the destination then that would break the cheap copy
 feature of Subversion's copy-on-write filesystem.

Anyway the URL of every file should be updated, shouldn't it?
At least there is an inconsistency, shown by the following script:


#!/bin/sh

set -ex

svn --version

mkdir my-test-svn
cd my-test-svn

svnadmin create svn
root=file://`pwd`/svn
svn co $root wc
cd wc

svn mkdir dir1
svn ci -m 'add dir1'

printf '\044Header\044\n'  dir1/file
svn add dir1/file
svn ps svn:keywords Header dir1/file
svn ci -m 'add dir1/file'
cat dir1/file
svn cat $root/dir1/file@2

svn up
svn mv dir1 dir2
svn ci -m 'mv dir1 dir2'
svn info dir2/file
cat dir2/file
svn cat $root/dir2/file@3

cd ../..
rm -rf my-test-svn


The output ends with:

+ cat dir2/file
$Header: file:///tmp/my-test-svn/svn/dir1/file 2 2012-07-04 11:57:43Z vlefevre $
+ svn cat file:///tmp/my-test-svn/svn/dir2/file@3
$Header: file:///tmp/my-test-svn/svn/dir2/file 2 2012-07-04 11:57:43Z vlefevre $

file:///tmp/my-test-svn/svn/dir1/file@2 exists but this isn't the
real URL of the file.

file:///tmp/my-test-svn/svn/dir2/file@2 doesn't exist.

IMHO, the most intuitive Header string should have

  file:///tmp/my-test-svn/svn/dir2/file 3

(correct URL, and a revision that is valid for this URL).

Full output attached.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
+ svn --version
svn, version 1.7.5 (r1336830)
   compiled Jun 17 2012, 07:52:38

Copyright (C) 2012 The Apache Software Foundation.
This software consists of contributions made by many people; see the NOTICE
file for more information.
Subversion is open source software, see http://subversion.apache.org/

The following repository access (RA) modules are available:

* ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
  - handles 'http' scheme
  - handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
  - with Cyrus SASL authentication
  - handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
  - handles 'http' scheme
  - handles 'https' scheme

+ mkdir my-test-svn
+ cd my-test-svn
+ svnadmin create svn
+ pwd
+ root=file:///tmp/my-test-svn/svn
+ svn co file:///tmp/my-test-svn/svn wc
Checked out revision 0.
+ cd wc
+ svn mkdir dir1
A dir1
+ svn ci -m add dir1
Adding dir1

Committed revision 1.
+ printf \044Header\044\n
+ svn add dir1/file
A dir1/file
+ svn ps svn:keywords Header dir1/file
property 'svn:keywords' set on 'dir1/file'
+ svn ci -m add dir1/file
Adding dir1/file
Transmitting file data .
Committed revision 2.
+ cat dir1/file
$Header: file:///tmp/my-test-svn/svn/dir1/file 2 2012-07-04 11:57:43Z vlefevre $
+ svn cat file:///tmp/my-test-svn/svn/dir1/file@2
$Header: file:///tmp/my-test-svn/svn/dir1/file 2 2012-07-04 11:57:43Z vlefevre $
+ svn up
Updating '.':
At revision 2.
+ svn mv dir1 dir2
A dir2
D dir1
D dir1/file
+ svn ci -m mv dir1 dir2
Deleting   dir1
Adding dir2

Committed revision 3.
+ svn info dir2/file
Path: dir2/file
Name: file
Working Copy Root Path: /tmp/my-test-svn/wc
URL: file:///tmp/my-test-svn/svn/dir2/file
Repository Root: file:///tmp/my-test-svn/svn
Repository UUID: d7bbc0d2-1977-4b50-a670-ad8c8a0c08df
Revision: 3
Node Kind: file
Schedule: normal
Last Changed Author: vlefevre
Last Changed Rev: 2
Last Changed Date: 2012-07-04 13:57:43 +0200 (Wed, 04 Jul 2012)
Text Last Updated: 2012-07-04 13:57:43 +0200 (Wed, 04 Jul 2012)
Checksum: 074821f52733e39ab981a36c5e46748b8ebbbc75

+ cat dir2/file
$Header: file:///tmp/my-test-svn/svn/dir1/file 2 2012-07-04 11:57:43Z vlefevre $
+ svn cat file:///tmp/my-test-svn/svn/dir2/file@3
$Header: file:///tmp/my-test-svn/svn/dir2/file 2 2012-07-04 11:57:43Z vlefevre $
+ cd ../..
+ rm -rf my-test-svn

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-04 13:58:50 +0200, Johan Corveleyn wrote:
 I don't understand. It's pretty clear that 1.6 is incorrect, because
 it puts a different LCR value in the originating working copy than in
 the repository.

Yes, 1.6 is incorrect, but 1.6 might have a regression compared
to 1.4 and/or 1.5. Or 1.4 and/or 1.5 were also incorrect and the
bug have never really been fixed. 1.7 is also incorrect if you
consider the URL (see my reply to Philip Martin).

  The problem now is that the URL + the LastChangedRev together can
  no longer identify the file, because the LastChangedRev can now be
  incorrect as a peg revision.
 
 I'm afraid I don't follow you here either. Can you give a concrete example?

Well, this is more complex that I thought, and 1.7 really has an
inconsistency. My example is in my reply to Philip Martin.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-04 13:28:21 +0100, Philip Martin wrote:
 Vincent Lefevre vincent-...@vinc17.net writes:
  The output ends with:
 
  + cat dir2/file
  $Header: file:///tmp/my-test-svn/svn/dir1/file 2 2012-07-04 11:57:43Z 
  vlefevre $
  + svn cat file:///tmp/my-test-svn/svn/dir2/file@3
  $Header: file:///tmp/my-test-svn/svn/dir2/file 2 2012-07-04 11:57:43Z 
  vlefevre $
 
  file:///tmp/my-test-svn/svn/dir1/file@2 exists but this isn't the
  real URL of the file.
 
 That's a keyword expansion bug, a variation on issue 1975.
 
  file:///tmp/my-test-svn/svn/dir2/file@2 doesn't exist.
 
  IMHO, the most intuitive Header string should have
 
file:///tmp/my-test-svn/svn/dir2/file 3
 
 Where would the revision 3 come from?  LastChangedRev is 2.  That's what
 Subversion's cheap copy means.

Yes, but I meant that LastChangedRev could be 3 after a move.
I don't think this contradicts cheap copy: when doing

  svn cat file:///tmp/my-test-svn/svn/dir2/file@3

Subversion gets the file via a COPY node or something like that
(I don't know the exact internals) as the file hasn't changed,
and it could get LastChangedRev from it.

My point is that URL@LastChangedRev should always be a valid
reference to the file, and it should be equivalent to URL@HEAD
or just URL.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-04 14:26:07 +0200, Johan Corveleyn wrote:
 We now have two clearly separate discussions:
 
 - Inconsistencies in LCR between originating WC and repository. That's
 my main interest right now. I think those are clearly bugs (1.6 had
 one, and now 1.7 has another one it seems).
 
 - Whether or not LCR should be updated if a parent directory gets
 moved/renamed. That's more a discussion of specification, what's the
 desired behavior, ... That's a bit out of my personal scope right now,
 but maybe others will participate further in this discussion.

But how the inconsistencies should be fixed depends on the
specification. Specification should come first.

 You mentioned one concrete use-case which breaks down because of
 this: being able to identify a file with URL@LCR (I'm not sure if
 this is (or ever was) an intended use-case). Is there other behavior
 that depends on this?

Well, I don't see the point of the URL if the goal is not to
identify the file. And since files can be removed and added,
a peg revision is necessary.

But perhaps the URL could be changed to a URL with a peg revision,
which can be different from LCR.

 Now that I think about it: the current behavior of LCR not bumped by
 parent move/copy is there at least since 1.5.

Note that ViewVC behaves differently. See:

  https://gforge.inria.fr/scm/viewvc.php/tags/3.1.1/?root=mpfr

All the Rev's are the same. This is different to what one obtains with
svn ls -v. The svn version on the server is 1.6.12 (r955767).

 We have a 1.5 repository, and I'm pretty used to the fact that, when
 I look at a branch or tag with a repository browser, that I see the
 LastChangedRev and LastChangeAuthor and Date as being the last real
 modification of that file (not the revision which copied the file
 to the branch/tag). I kinda like that behavior...

I think that both kinds of information can be interesting. It is
interesting to see that files under a tag have not been modified
since the tag. But there could be a boolean saying whether the
file had its URL changed since the real file change.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-04 14:01:36 +0100, Philip Martin wrote:
 Vincent Lefevre vincent-...@vinc17.net writes:
 
  On 2012-07-04 13:28:21 +0100, Philip Martin wrote:
  
  Where would the revision 3 come from?  LastChangedRev is 2.  That's what
  Subversion's cheap copy means.
 
  Yes, but I meant that LastChangedRev could be 3 after a move.
  I don't think this contradicts cheap copy: when doing
 
svn cat file:///tmp/my-test-svn/svn/dir2/file@3
 
  Subversion gets the file via a COPY node or something like that
  (I don't know the exact internals) as the file hasn't changed,
  and it could get LastChangedRev from it.
 
 No, it doesn't follow a copy.  dir2@3 has a reference to the the same
 child object as dir1@2.  dir2/file@3 is exactly the same object as
 dir1/file@2.  That's what a cheap copy means.

But how does Subversion know that dir2/file@3 is the same object as
dir1/file@2? It must follow some reference, and if a LastChangedRev
is associated to each reference in the internal data structures,
Subversion will get the correct revision. At worst, you will only
lose a constant factor in time and memory.

  My point is that URL@LastChangedRev should always be a valid
  reference to the file, and it should be equivalent to URL@HEAD
  or just URL.
 
 That's not the way it works.  I think you want:
 
-rLastChangedRev URL@Revision
 
 i.e. take the current URL@Revision and go back to LastChangedRev.

Well, I think that the Header keyword is really confusing because it
just gives URL and LastChangedRev, while if a URL and revision
are given together, one would expect URL and Revision (or some
equivalent value).

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-06-27 16:33:23 +0200, Johan Corveleyn wrote:
 I noticed this while investigating (other) problems with incorrect
 LastChangedRev's (subtle form of working copy corruption):

It seems to be the same thing I had reported here:

http://mail-archives.apache.org/mod_mbox/subversion-dev/201206.mbox/%3C20120619010211.GE5263%40xvii.vinc17.org%3E

and

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=678075

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incorrect LastChangedRev in working copy after committing directory move + modified file in subdir

[Vincent Lefevre]

On 2012-07-03 13:15:33 +0200, Johan Corveleyn wrote:
 No, it's not the same issue (the issue you reported is, let's say,
 open for discussion, and I don't think the 1.7 behavior is wrong ...
 it's just different from 1.6).

OK, I actually focused on svn info .../subdir, where you get
Last Changed Rev: 2 with svn 1.7.5 while with svn 1.6.x, one
would have got 3 (because the subdir parent moved at rev 3),
and this corresponded to what I reported.

Now in your case, you also modify test.txt in subdir. After some
tests, it seems that the LastChanged* metadata of a directory are
updated whenever something has changed below the directory (except
in particular cases as you reported). This is something I didn't
notice before, even though it is the same behavior as with 1.6.x.
So, according to this behavior, one would expect 3.

I would say that the main problem is that the LastChanged* metadata
are underspecified. The Subversion book doesn't seem to specify
anything except for keyword substitution, where it says:

  This keyword describes the last known revision in which this file
  changed in the repository, and looks something like $Revision: 144 $.
  It may also be specified as LastChangedRevision or Rev.

But a change of the contents of a file isn't a change of the directory
that contains it (neither of any of its ancestors). So, the current
behavior of svn info on directories would be incorrect in most cases.

 The issue I'm reporting in this thread is that the LastChangedRev
 differs between 'svn info subdir' and 'svn info
 url-of-subdir@workingrev-of-subdir'. That seems wrong, it means that
 the working copy (from where the move was committed) has a
 LastChangedRev that's different from the repository and from any other
 working copy that's checked out (from that same working revision).

Actually I would say that svn info subdir is correct, and all the
other cases are incorrect.

 Note that this is only the LastChangedRev of the intermediate subdir
 which I'm talking about, so I don't think there's an impact on Keyword
 Substitution.

Yes, the issue is specific to directories, for which there are no
possible keyword substitutions.

-- 
Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/
100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)