Re: Error E160016 "... is not a directory in filesystem ..."

[Stefan Sperling]

On Fri, Nov 03, 2017 at 01:33:03PM +0100, Vincent Lefevre wrote:
> As I thought, I couldn't reproduce the bug with such a sample
> repository.
> 
> But I've found that the bug is fixed in the trunk. I could bisect
> to search for the fix, and this is the following:
> 
> 
> r1795116 | stefan2 | 2017-05-14 22:22:49 +0200 (Sun, 14 May 2017) | 17 lines
> 
> Fix issue #4677.

Great! Thanks for tracking it down.

> There is a comment saying:
> 
> "Backport for 1.9.6 is yet to be approved."
> 
> Could it really be backported?

It was already queued in the 1.9.x STATUS file and was just waiting for
one more vote. I have cast the missing vote in http://svn.apache.org/r1814174

So barring unexpected events, this should be part of the next 1.9.x
release (I cannot give you an ETA on that, though).

Re: Error E160016 "... is not a directory in filesystem ..."

[Branko Čibej]

On 03.11.2017 13:33, Vincent Lefevre wrote:
> There is a comment saying:
>
> "Backport for 1.9.6 is yet to be approved."
>
> Could it really be backported?

It's in the queue for backport, it just hasn't made it through yet ...

-- Brane

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

On 2017-11-03 10:53:07 +0100, Vincent Lefevre wrote:
> Unfortunately, currently I can't. A few days ago, I tried to reproduce
> it with similar changes in a sample repository, but this didn't trigger
> the bug. I'll try again from information based on strace, i.e. on the
> fact that "perso" was moved in the past. But I fear that this may be
> more complex (otherwise the bug would have already been found).

As I thought, I couldn't reproduce the bug with such a sample
repository.

But I've found that the bug is fixed in the trunk. I could bisect
to search for the fix, and this is the following:


r1795116 | stefan2 | 2017-05-14 22:22:49 +0200 (Sun, 14 May 2017) | 17 lines

Fix issue #4677.

When non-exitent paths are allowed as a result of a DAG-walk
(as opposed to creating a "path not found" error on those),
we may encounter file nodes in the "parent" path.  Those are
not an error but simply an indication that there won't be any
sub-paths.

* subversion/libsvn_fs_fs/tree.c
  (open_path): If n/a paths are allowed, non-dir parents are, too.

* subversion/libsvn_fs_x/dag_cache.c
  (svn_fs_x__get_dag_path): Same.

* subversion/tests/libsvn_fs/fs-test.c
  (closest_copy_test_svn_4677): Add regression test.
  (test_funcs): Register new test.


https://issues.apache.org/jira/browse/SVN-4677 contains:

This bug was introduced in
http://svn.apache.org/viewvc?view=revision=1586953 , is
present in all releases since 1.9.0, and can be fixed by reverting
that commit.

And the test shows a similar error:

Updating '.':
svn: E160016: Failure opening '/b/fa/fb'
svn: E160016: '/b/fa' is not a directory in filesystem 
'1d2dc934-16b4-452c-a607-a6080764124e'

There is a comment saying:

"Backport for 1.9.6 is yet to be approved."

Could it really be backported?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Stefan Sperling]

On Fri, Nov 03, 2017 at 10:53:07AM +0100, Vincent Lefevre wrote:
> On 2017-11-03 10:01:26 +0100, Stefan Sperling wrote:
> > I agree this looks like a bug. However, it is unclear how to reproduce it.
> > 
> > Can you provide a sample repository (or shell script) which triggers the 
> > bug?
> 
> Unfortunately, currently I can't. A few days ago, I tried to reproduce
> it with similar changes in a sample repository, but this didn't trigger
> the bug. I'll try again from information based on strace, i.e. on the
> fact that "perso" was moved in the past. But I fear that this may be
> more complex (otherwise the bug would have already been found).
> 
> However, I could do some tests if Subversion could log what it is
> doing at the low level.

Or run it in gdb(4) and see if you can find out what's going wrong.

If you wish to instrument the source with debug prints, just put an
SVN_DBG(("foo")) macro invocation anywhere (supports format strings).

> I suppose I could also try to find which commit triggered the bug.

Yes, that would help.

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

On 2017-11-03 10:01:26 +0100, Stefan Sperling wrote:
> I agree this looks like a bug. However, it is unclear how to reproduce it.
> 
> Can you provide a sample repository (or shell script) which triggers the bug?

Unfortunately, currently I can't. A few days ago, I tried to reproduce
it with similar changes in a sample repository, but this didn't trigger
the bug. I'll try again from information based on strace, i.e. on the
fact that "perso" was moved in the past. But I fear that this may be
more complex (otherwise the bug would have already been found).

However, I could do some tests if Subversion could log what it is
doing at the low level.

I suppose I could also try to find which commit triggered the bug.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Stefan Sperling]

On Fri, Nov 03, 2017 at 12:44:03AM +0100, Vincent Lefevre wrote:
> On 2017-11-03 00:18:59 +0100, Vincent Lefevre wrote:
> > cventin:~> svn ls -r99809 
> > file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
> > svn: E160016: Failure opening '/perso/tcl/fidelite'
> > svn: E160016: '/perso/tcl' is not a directory in filesystem 
> > '99759db8-4ec0-0310-8bf9-df86780d22d8'
> 
> For this one, valgrind doesn't signal any issue. And with strace,
> I can see that:
> 
> .../db/revs/99/99809 is opened 5 times.
> .../db/revs/99/99807 is opened 3 times.
> .../db/revs/52/52653 is opened 5 times.
> 
> r99807 is the revision where the /perso/tcl directory and the
> /perso/tcl/fidelite file were added.
> 
> r52653 is the revision where the /misc/perso directory was renamed
> to /perso.

I agree this looks like a bug. However, it is unclear how to reproduce it.

Can you provide a sample repository (or shell script) which triggers the bug?

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

On 2017-11-03 00:18:59 +0100, Vincent Lefevre wrote:
> cventin:~> svn ls -r99809 
> file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
> svn: E160016: Failure opening '/perso/tcl/fidelite'
> svn: E160016: '/perso/tcl' is not a directory in filesystem 
> '99759db8-4ec0-0310-8bf9-df86780d22d8'

For this one, valgrind doesn't signal any issue. And with strace,
I can see that:

.../db/revs/99/99809 is opened 5 times.
.../db/revs/99/99807 is opened 3 times.
.../db/revs/52/52653 is opened 5 times.

r99807 is the revision where the /perso/tcl directory and the
/perso/tcl/fidelite file were added.

r52653 is the revision where the /misc/perso directory was renamed
to /perso.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

[Vincent Lefevre]

[Moving to dev]

On 2017-11-01 17:05:24 +0100, Vincent Lefevre wrote:
> Also, this is not consistent, unless future commits have an influence
> on the following command:
> 
> joooj:~> svn ls -r99809 
> file:///srv/d_joooj/home/svn/root/perso/tcl/fidelite@99808
> svn: E160016: Failure opening '/perso/tcl/fidelite'
> svn: E160016: '/perso/tcl' is not a directory in filesystem 
> '99759db8-4ec0-0310-8bf9-df86780d22d8'
> 
> ioooi:~> svn ls -r99809 
> file:///srv/d_ioooi/home/svn/root/perso/tcl/fidelite@99808
> fidelite
> 
> FYI, ioooi is the old server, which I migrated with dump+load to
> joooj at revision r102683. So, I should have obtained the same
> result; the one given by ioooi is the expected one.

This is definitely a bug, and a regression in 1.9.

I did the following test:

1. On a Debian/unstable machine (which has Subversion 1.9.7 r1800392),
I created a format 3 FSFS repository and loaded the dump.

2. I tested the issue:

cventin:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

cventin:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

cventin:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
svn: E160016: Failure opening '/perso/tcl/fidelite'
svn: E160016: '/perso/tcl' is not a directory in filesystem 
'99759db8-4ec0-0310-8bf9-df86780d22d8'

cventin:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
svn: E160016: Failure opening '/perso/tcl/fidelite'
svn: E160016: '/perso/tcl' is not a directory in filesystem 
'99759db8-4ec0-0310-8bf9-df86780d22d8'

So, this is buggy.

3. I copied the repository to a Debian/jessie machine (which has
Subversion 1.8.10 r1615264).

4. I tested the issue on this machine:

tomate:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

tomate:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

tomate:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

tomate:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

Everything is fine with the old version.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)