Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
> > I thought that the API is > > defined for UTF8-strings, so I can understand about using urlencode for > > [%\/?&] - but for "normal", printable UTF8 characters? > > Well, the description of svn_path_canonicalize[0] leaves vague the full > specification of what transformations will be applied: ... > > > I'm inclined to close this as the API being used incorrectly. > > So, I should just make FSVS pass eg. "ö" as "%C3%B6" instead? > > I'd suggest using svn_path_canonicalize (or one the more granular > replacements, depending on the minimal version of SVN you want to > impose). Then you know you're using a properly formed path. Done, see http://fsvs.tigris.org/source/browse/fsvs?view=rev=2487 Thanks for your help!!
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
Hi James, thank you very much for your patience! > > > Granted, the assert is new but it's just asserting the contract of the > > > APIs. The URIs were always supposed to be canonicalized. > > Yeah, and I even have code that deals with a slash at the end and so on. > > > > The other idea is that the requirements have changed - perhaps an UTF8 URL > > like > > file:///tmp/fsvs-test-1044/repos/trunk/ä-30305/ö-13157/§-2186 > > isn't allowed anymore, but the non-ASCII characters have to be encoded? > > From what I can tell, that was always supposed to be the case. What was? That non-ASCII needs to be encoded? I thought that the API is defined for UTF8-strings, so I can understand about using urlencode for [%\/?&] - but for "normal", printable UTF8 characters? > I'm inclined to close this as the API being used incorrectly. So, I should just make FSVS pass eg. "ö" as "%C3%B6" instead?
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
On Sun, Jan 10, 2016 at 12:07:15PM +0100, Philipp Marek wrote: > Hi James, > > thank you very much for your patience! > > > > > > Granted, the assert is new but it's just asserting the contract of the > > > > APIs. The URIs were always supposed to be canonicalized. > > > Yeah, and I even have code that deals with a slash at the end and so on. > > > > > > The other idea is that the requirements have changed - perhaps an UTF8 > > > URL > > > like > > > file:///tmp/fsvs-test-1044/repos/trunk/ä-30305/ö-13157/§-2186 > > > isn't allowed anymore, but the non-ASCII characters have to be encoded? > > > > From what I can tell, that was always supposed to be the case. > What was? That non-ASCII needs to be encoded? Yes. > I thought that the API is > defined for UTF8-strings, so I can understand about using urlencode for > [%\/?&] - but for "normal", printable UTF8 characters? Well, the description of svn_path_canonicalize[0] leaves vague the full specification of what transformations will be applied: This involves collapsing redundant "/./" elements, removing multiple adjacent separator characters, removing trailing separator characters, and possibly other semantically inoperative transformations. [0]: https://subversion.apache.org/docs/api/latest/svn__path_8h.html#a71d5716a18e04df175633a65ddac7809 Testing shows that the canonical version of the path does indeed encode the UTF-8 characters: $ python Python 2.7.11 (default, Dec 9 2015, 00:29:25) [GCC 5.3.1 20151205] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from svn import core >>> core.svn_path_canonicalize('file:///tmp/fsvs-test-1044/repos/trunk/ä-30305/ö-13157/§-2186') 'file:///tmp/fsvs-test-1044/repos/trunk/%C3%A4-30305/%C3%B6-13157/%C2%A7-2186' > > I'm inclined to close this as the API being used incorrectly. > So, I should just make FSVS pass eg. "ö" as "%C3%B6" instead? I'd suggest using svn_path_canonicalize (or one the more granular replacements, depending on the minimal version of SVN you want to impose). Then you know you're using a properly formed path. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
On Fri, Jan 08, 2016 at 08:25:57AM +0100, Philipp Marek wrote: > > Granted, the abort needs to be fixed, Or not. The library is asserting the expected semantics of its API. The semantics didn't change, just how strictly they were being checked did. > > Ok, during SVN 1.7 development (r957893) moved a bunch of code from > > svn_ra_local__split_URL to a new function, > > svn_uri_get_dirent_from_file_url. When that happened, the assert you're > > hitting was added: > > > > assert(svn_uri_is_canonical(url, pool)); > > > > You're not canonicalizing the URL before passing it into svn_ra_open(). > > You probably want to use svn_path_canonicalize() (or if you can accept a > > minimum SVN version of 1.6, one of svn_dirent_canonicalize, > > svn_uri_canonicalize, svn_relpath_canonicalize) in url__parse instead of > > manually trying to make it canonical. > > > > Granted, the assert is new but it's just asserting the contract of the > > APIs. The URIs were always supposed to be canonicalized. > Yeah, and I even have code that deals with a slash at the end and so on. > > > The other idea is that the requirements have changed - perhaps an UTF8 URL > like > file:///tmp/fsvs-test-1044/repos/trunk/ä-30305/ö-13157/§-2186 > isn't allowed anymore, but the non-ASCII characters have to be encoded? >From what I can tell, that was always supposed to be the case. I'm inclined to close this as the API being used incorrectly. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
On Mon, Dec 28, 2015 at 11:27:28AM +0100, Ph. Marek wrote: > During a run with my automated test suite for FSVS I noticed that some > things that previously worked do not any more. > I can't really give a hard range of SVN versions, but I guess 2 or 3 years > ago that wasn't a problem yet ;/ Hmmm. I tested all the way back to SVN 1.5 and I either get the abort you're reporting or a failure trying to setup the working copy in order to run the test. $ make BINARY=../src/fsvs TEST_LIST=037\* -C tests make: Entering directory '/home/jamessan/src/debian.org/fsvs/fsvs-1.2.6/tests' test -d /tmp/fsvs-test-1000 || mkdir -p /tmp/fsvs-test-1000 test -d /tmp/fsvs-test-1000/waa || mkdir -p /tmp/fsvs-test-1000/waa test -d /tmp/fsvs-test-1000/conf || mkdir -p /tmp/fsvs-test-1000/conf Preparing default repository. An error occurred: Couldn't open a repository (180001) in url__open_session: svn_ra_open("file:///tmp/fsvs-test-1000/repos/trunk"): Unable to connect to a repository at URL 'file:///tmp/fsvs-test-1000/repos/trunk' make[3]: *** [prepare_wc] Error 2 An error occurred: Couldn't open a repository (180001) in url__open_session: svn_ra_open("file:///tmp/fsvs-test-1000/repos/trunk"): Unable to connect to a repository at URL 'file:///tmp/fsvs-test-1000/repos/trunk' make[3]: *** [prepare_wc] Error 2 make[2]: *** [prepare_wcs] Error 2 make[1]: *** [prepare_clean] Error 2 Makefile:166: recipe for target '/tmp/fsvs-test-1000/default-repos.dump' failed make: *** [/tmp/fsvs-test-1000/default-repos.dump] Error 2 make: Leaving directory '/home/jamessan/src/debian.org/fsvs/fsvs-1.2.6/tests' I didn't get a single successful run of the test. Granted, the abort needs to be fixed, but not being able to run the test makes it hard to bisect where things broke. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
On Thu, Jan 07, 2016 at 07:33:38PM -0500, James McCoy wrote: > On Mon, Dec 28, 2015 at 11:27:28AM +0100, Ph. Marek wrote: > > During a run with my automated test suite for FSVS I noticed that some > > things that previously worked do not any more. > > I can't really give a hard range of SVN versions, but I guess 2 or 3 years > > ago that wasn't a problem yet ;/ > > Hmmm. I tested all the way back to SVN 1.5 and I either get the abort > you're reporting or a failure trying to setup the working copy in order > to run the test. Ok, during SVN 1.7 development (r957893) moved a bunch of code from svn_ra_local__split_URL to a new function, svn_uri_get_dirent_from_file_url. When that happened, the assert you're hitting was added: assert(svn_uri_is_canonical(url, pool)); You're not canonicalizing the URL before passing it into svn_ra_open(). You probably want to use svn_path_canonicalize() (or if you can accept a minimum SVN version of 1.6, one of svn_dirent_canonicalize, svn_uri_canonicalize, svn_relpath_canonicalize) in url__parse instead of manually trying to make it canonical. Granted, the assert is new but it's just asserting the contract of the APIs. The URIs were always supposed to be canonicalized. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
Hi James, > $ make BINARY=../src/fsvs TEST_LIST=037\* -C tests ... > An error occurred: Couldn't open a repository (180001) > in url__open_session: > svn_ra_open("file:///tmp/fsvs-test-1000/repos/trunk"): Unable to connect to a > repository at URL 'file:///tmp/fsvs-test-1000/repos/trunk' > make[3]: *** [prepare_wc] Error 2 ... > I didn't get a single successful run of the test. Just sitting in src/ and running "make run-tests TEST_LIST=037*" doesn't work for you? Is the repository in "/tmp/fsvs-test-1000/repos/" created for you? Hmmm, what could be different? I've got /tmp on a tmpfs. > Granted, the abort needs to be fixed, but not being able to run the test > makes it hard to bisect where things broke. Yeah, right. > Ok, during SVN 1.7 development (r957893) moved a bunch of code from > svn_ra_local__split_URL to a new function, > svn_uri_get_dirent_from_file_url. When that happened, the assert you're > hitting was added: > > assert(svn_uri_is_canonical(url, pool)); > > You're not canonicalizing the URL before passing it into svn_ra_open(). > You probably want to use svn_path_canonicalize() (or if you can accept a > minimum SVN version of 1.6, one of svn_dirent_canonicalize, > svn_uri_canonicalize, svn_relpath_canonicalize) in url__parse instead of > manually trying to make it canonical. > > Granted, the assert is new but it's just asserting the contract of the > APIs. The URIs were always supposed to be canonicalized. Yeah, and I even have code that deals with a slash at the end and so on. The other idea is that the requirements have changed - perhaps an UTF8 URL like file:///tmp/fsvs-test-1044/repos/trunk/ä-30305/ö-13157/§-2186 isn't allowed anymore, but the non-ASCII characters have to be encoded? Thank you for your help!
Bug#809209: libsvn_subr-1.so.1: svn: E235000: dirent_uri.c:2335: assertion
Package: libsvn1 Version: 1.9.2-3+b1 Severity: normal File: /usr/lib/x86_64-linux-gnu/libsvn_subr-1.so.1 Original subject before shortening: /usr/lib/x86_64-linux-gnu/libsvn_subr-1.so.1: svn: E235000: In file '.../dirent_uri.c' line 2335: assertion failed (svn_uri_is_canonical(url, pool)) During a run with my automated test suite for FSVS I noticed that some things that previously worked do not any more. I can't really give a hard range of SVN versions, but I guess 2 or 3 years ago that wasn't a problem yet ;/ Basically, my code does an "svn_ra_open()" with an URL of "file:///tmp/fsvs-test-1044/repos/trunk/ä-30305/ö-13157/§-2186"; the repository base is (easily guessed ;) at "file:///tmp/fsvs-test-1044/repos/". I guess that the non-ASCII characters in there are not parsed correctly (any more). The stack trace shows this, with the last three being in FSVS: Program received signal SIGABRT, Aborted. 0x7684a657 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x7684a657 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7684ba2a in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x77b0558e in svn_error_abort_on_malfunction () from /usr/lib/x86_64-linux-gnu/libsvn_subr-1.so.1 #3 0x77b03a45 in svn_uri_get_dirent_from_file_url () from /usr/lib/x86_64-linux-gnu/libsvn_subr-1.so.1 #4 0x7587b74d in svn_ra_local__split_URL () from /usr/lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1 #5 0x7587af3d in ?? () from /usr/lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1 #6 0x77698d43 in svn_ra_open4 () from /usr/lib/x86_64-linux-gnu/libsvn_ra-1.so.1 #7 0x77695c3d in svn_ra_open3 () from /usr/lib/x86_64-linux-gnu/libsvn_ra-1.so.1 #8 0x77695c66 in svn_ra_open2 () from /usr/lib/x86_64-linux-gnu/libsvn_ra-1.so.1 #9 0x77695d26 in svn_ra_open () from /usr/lib/x86_64-linux-gnu/libsvn_ra-1.so.1 #10 0x0040b88e in url__open_session (session=session@entry=0x0, missing_dirs=missing_dirs@entry=0x7fffdf98) at url.c:1103 #11 0x0040f5c3 in ci__work (root=0x7fffe080, argc=0x0, argv=0x7fffe278) at commit.c:1138 #12 0x00404aa8 in main (argc=, args=0x7fffe268, env=) at fsvs.c:1408 The repository exists, but has no content yet. FSVS tries to open this RA path; previously it would simply get ENOENT back, so it could determine which intermediate directories it had to create - but now that an assertion is triggered, it won't get to that point any more, but just "crashes" before. In case that's of interest, this is the "037_mkdir_base" test in FSVS. Thank you! -- System Information: Debian Release: stretch/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable'), (500, 'stable'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 4.2.0-1-amd64 (SMP w/4 CPU cores) Locale: LANG=de_AT.UTF-8, LC_CTYPE=de_AT.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages libsvn1:amd64 depends on: ii libapr11.5.2-3 ii libaprutil11.5.4-1+b1 ii libc6 2.21-4 ii libdb5.3 5.3.28-11 ii libexpat1 2.1.0-7 ii libldap-2.4-2 2.4.42+dfsg-2 ii libsasl2-2 2.1.26.dfsg1-14+b1 ii libserf-1-11.3.8-1+b1 ii libsqlite3-0 3.9.2-1 ii zlib1g 1:1.2.8.dfsg-2+b1 libsvn1:amd64 recommends no packages. libsvn1:amd64 suggests no packages. -- no debconf information -- : Ing. Philipp Marek : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com : DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.