Re: "svn add" crashes in case of missing magic.mgc
Guten Tag Stefan Sperling, am Montag, 8. Juli 2019 um 20:36 schrieben Sie: > As a workaround, you can disable use of libmagic in SVN's configuration. > In ~/.subversion/config, add these lines: > [miscellany] > enable-magic-file = no What would be the consequences of disabling? Do added files get some common fallback mime like octet-stream or none or forces "add" me to provide one? Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: "svn add" crashes in case of missing magic.mgc
Guten Tag Stefan Sperling, am Montag, 8. Juli 2019 um 18:10 schrieben Sie: > SVN only calls the magic_open() function provided by libmagic. > It checks the return value and skips mime-type detection via > libmagic if an error occurs. I believe that, at the time, this > code was tested with and without a magic database file and both > cases worked. But I don't think we have regression tests for this... It might perfectly be that there's something strange with that SVN client, because even if I make the file magic.mgc available to SVN by additionally linking in one of the checked places or using the environment variable "MAGIC", I see a lot of warnings about that file being invalid: > /usr/share/file/misc/magic.mgc, 92: Warning: offset `Ancient_Realms' invalid > /usr/share/file/misc/magic.mgc, 93: Warning: offset `Armagon's_Lair' invalid > /usr/share/file/misc/magic.mgc, 94: Warning: offset `Into_The_Flood' invalid > /usr/share/file/misc/magic.mgc, 95: Warning: offset `Slaughterhouse' invalid > ' invalide/file/misc/magic.mgc, 96: Warning: offset ` > ' invalide/file/misc/magic.mgc, 102: Warning: offset `METADATA > /usr/share/file/misc/magic.mgc, 103: Warning: offset `' invalid > /usr/share/file/misc/magic.mgc, 106: Warning: offset `' invalid > ' invalide/file/misc/magic.mgc, 108: Warning: offset `MozFASL > /usr/share/file/misc/magic.mgc, 109: Warning: offset `' invalid > /usr/share/file/misc/magic.mgc, 111: Warning: offset `' invalid > /usr/share/file/misc/magic.mgc, 112: Warning: offset `' invalid > /usr/share/file/misc/magic.mgc, 125: Warning: offset ` REMARK' invalid > /usr/share/file/misc/magic.mgc, 126: Warning: offset `' invalid > ' invalide/file/misc/magic.mgc, 127: Warning: offset `TypeLib > /usr/share/file/misc/magic.mgc, 128: Warning: offset `' invalid > /usr/share/file/misc/magic.mgc, 129: Warning: offset `' invalid Looks like the linked libmagic is incompatible anyway with the provided file. > If possible please try to reproduce with a newer version. [...] > Your best next step is to pin down the exact location of the > crash with a debugger (either live or with a core file). > Is SVN crashing or is libmagic crashing? I'll keep that in mind, but have very limited time currently for further research. No idea where to get symbol files from and stuff like that... > (gdb) run add /tmp/svn_crash/test.txt > Starting program: /volume1/@entware-ng/opt/bin/svn add /tmp/svn_crash/test.txt > warning: Unable to find libthread_db matching inferior's thread library, > thread debugging will not be available. > > Program received signal SIGABRT, Aborted. > 0x73d240f8 in raise () from /opt/lib/libc.so.6 > (gdb) bt > #0 0x73d240f8 in raise () from /opt/lib/libc.so.6 > #1 0x73d2552a in abort () from /opt/lib/libc.so.6 > #2 0x74711dd5 in ?? () from /opt/lib/libmagic.so.1 > #3 0x74716000 in ?? () from /opt/lib/libmagic.so.1 > #4 0x75c652ed in svn_magic.init () from /opt/lib/libsvn_subr-1.so.0 > #5 0x77b83600 in svn_client_add5 () from > /opt/lib/libsvn_client-1.so.0 > #6 0x004077af in ?? () > #7 0x0041c39a in ?? () > #8 0x004075b3 in ?? () > #9 0x73d11711 in __libc_start_main () from /opt/lib/libc.so.6 > #10 0x00407629 in ?? () Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
"svn add" crashes in case of missing magic.mgc
Hi all,
I recently set up a new Synology NAs and tried to version some custom
scripts. While some SVN client was already available and checking out
an empty directory worked as expected, adding some file to the working
copy didn't. There wasn't any error by default, it seemed that adding
was successful, but it wasn't, instead the working copy was locked.
> svn add dasikomp_buero_hm
> svn st
> L .
> ? @eaDir
> ? dasikomp_buero_hm
[...]
Explicitly asking for the last error showed the problem:
> svn add %f && echo $@
> sh: line 1: 20166 Aborted (core dumped) svn add "test.txt"
Making strace available[1] lead to the root cause being the file
magic.mgc not found in usual places:
> strace svn add %f
[...]
> stat("/root/.magic.mgc", 0x7fff5f129370) = -1 ENOENT (No such file or
> directory)
> stat("/root/.magic", 0x7fff5f129370)= -1 ENOENT (No such file or
> directory)
> access("/opt/share/misc/magic.mime.mgc", R_OK) = -1 ENOENT (No such file or
> directory)
> open("/opt/share/misc/magic.mgc", O_RDONLY) = -1 ENOENT (No such file or
> directory)
> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
> tgkill(21192, 21192, SIGABRT) = 0
> --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=21192, si_uid=0} ---
Instead, the file was placed in the following path, which is a symlink
only as well:
> /usr/share/file/misc/magic.mgc ->
> /var/packages/DiagnosisTool/target/usr/share/file/misc/magic.mgc
The link target is part of DiagnosisTools, which is interesting
because that's exactly what has been installed using [1] to make
strace available. Without that package I didn't find any other
suitable file on the NAS.
Some things coming into my mind:
1. Should "svn add" really crash silently?
2. Should the symlink be recognized and tried like the other pathes?
3. Is path handling even something SVN cares about on it's own vs.
libmagic or else?
The installed SVN-client:
> svn, version 1.9.7 (r1800392)
>compiled Jan 11 2018, 08:42:15 on x86_64-openwrt-linux-gnu
Is that worth filing a bug or fixed or known or ...? Thanks!
[1]: https://www.synology-wiki.de/index.php/Synogear:_weitere_Tools
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...05151- 9468- 55
Fax...05151- 9468- 88
Mobil..0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
mod_dav_dvn+SVNParentPath with non-repo subdirs
Hi all, I'm hosting SVN-repos using svnserve and by providing the argument "--root" with the parent dir of all my repos. In the past, all repos have been placed as a direct child of the given path, but recently I refactored that to manage repos by customers, topics and stuff. This works pretty easy with svnserve as it simply forwards all URLs given by clients into the file system, so one can really simply move repos into additional subdirs and those get published instantly. This doesn't work with mod_dav_svn: > Using this syntax, Apache will delegate the handling of all URLs > whose path portions begin with /svn/ to the Subversion DAV provider, > which will then assume that any items in the directory specified by > the SVNParentPath directive are actually Subversion repositories. http://svnbook.red-bean.com/en/1.8/svn.serverconfig.httpd.html https://stackoverflow.com/questions/2701915/svn-multiple-repositories-in-subfolders I was unable to find reasons and discussions about why things are implemented differently, if changes have already been suggested etc. Currently I'm placing a lot "SVNParentPath" in the config of my httpd, but would be great if one could reduce that in future. So, is there a reason why mod_dav_svn doesn't support additional subdirs? Is that likely to be changed? Is it likely that svnserve will stop supporting addiitonal child dirs? Thanks! Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Subversion's community health
Guten Tag Karl Fogel, am Freitag, 14. Juni 2019 um 18:16 schrieben Sie: > I just put out an informal Twitter poll to get a sense of how > people are using Subversion these days: > https://twitter.com/kfogel/status/1139559630059843586 > (Or at least, to get a sense of how Twitter users who happen to see > my tweet are using Subversion these days :-) .) Asking the same here and on the users list might make sense as well. I pretty much have the same use case like you, mostly with deployment to different customers using per-product SVN-repos. authz is simply necessary for us to restrict customers access to only their own tags. Additionally, the usage scenario of SVN better reflects what is needed with my customers: It doesn't make sense to allow them things like local branches to e.g. heavily customize the deployment and they don't even want to do that. Instead, if they change configs or such, that needs to be considered during updates anyway and SVN forces me to do so. That's simply a service to the customers making their life a bit easier. Some of them don't even understand why they should commit simple config changes using a reasonable log message for long term benefit, so run into conflicts because of incompatible local changes from time to time. Those customers wouldn't be able to work any better with e.g. GIT and things like commit vs. push and stuff. Running lots of different GIT-repos per product per-customer wouldn't make my life easier as well. I'm not necessarily such a big fan of local-only-branches possible with GIT for companies as well, because in my opinion, everything employees do should be available to the company mostly. Of course people don't necessarily need to commit using SVN as well, but that's refusing to work in the end. OTOH, for OSS-libs and -apps, Git and GitHub make life a lot easier. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: mod_dav_svn of SVN 1.9.3 doesn't seem to calc repo names from URLs with spaces properly.
Guten Tag Thorsten Schöning, am Freitag, 14. Juni 2019 um 12:12 schrieben Sie: > And I think that's the problem: The matched location is forwarded > using a space to mod_dav_svn, which tries to calculate the repo name > from the request and that request most likely containes %20 instead of > the space. So calculation simply fails: I'm not so sure about that anymore, because it seems the "Location" configured in httpd gets forwarded to mod_authz_svn and that seems to handle HEX-encoding as needed: > conf->base_path = svn_urlpath__canonicalize(d, p); https://github.com/apache/subversion/blob/1.9.3/subversion/mod_authz_svn/mod_authz_svn.c#L121 > uri = svn_fspath__canonicalize(uri, pool); > /* Do a little dance to normalize hex encoding. */ > uri = svn_path_uri_decode(uri, pool); > uri = svn_path_uri_encode(uri, pool); https://github.com/apache/subversion/blob/1.9.3/subversion/libsvn_subr/dirent_uri.c#L2622 This seems the request might contain the space instead of %20, but I have some mod_perl-handler on my own and am somewhat sure that that contained %20 instead of spaces. At least there's code in there unescaping an URI of a request... But doesn't change that much overall, I'm still unabale to tell mod_dav_svn to use the same URLs. > But I couldn't find anything simple like using spaces in URLs to > repos. I'm more convinced now that space is the correct thing to configure instead of %20 in "Location". Not only because it only works this way, but as well because such a discussion is more likely for mod_rewrite and in those case spaces are to be used as well always. https://stackoverflow.com/questions/410811/mod-rewrite-with-spaces-in-the-urls > Any workarounds you can think of with being able to keep the space in > the URL? I tried simply using a rewrite rule to internally change " " to "_", and while that works to properly forward requests containing "%20" to mod_dav_svn, the latter fails again. While I can authenticate successfully, SVN doesn't error out because of missing authz files and the client even starts creating the .svn-dir, a checkout doesn't work in the end because of the following errors: > Could not parse 'src-path' URL. [500, #190001] > Unusable URI: it does not refer to this repository [500, #190001] So internally replacing parts of the URL is not an option. Sending a redirect to the client instead seems to work: > RewriteRule "^(svn/Bin/LSG) (BE-BB/.+)$" "$1_$2" [L] vs. > RewriteRule "^(svn/Bin/LSG) (BE-BB/.+)$" "$1_$2" [R=301,L] It seems the only option currently is really to avoid the spaces in URLs containing the repo name. While keeping them in the configured SVNParentPath still works! Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
mod_dav_svn of SVN 1.9.3 doesn't seem to calc repo names from URLs with spaces properly.
Hi all, I'm hosting some SVN-repos using mod_dav_svn and am running into problems when using URLs with spaces. My approach is simply to have a directory structure of SVN-repos grouped by customers or some topic and for each of those groups I want to configure a corresponding URL as entry point into mod_dav_svn. Some of those groups contain spaces for historical reasons and because I want to use 1:1 paths in the file system as URLs, those spaces should be handled by httpd and mod_dav_svn as well. Have a look at the following config: > > DAV svn > SVNParentPath "/home/ams_svn_repos/Bin/LSG BE-BB" > SVNListParentPath On > SVNAdvertiseV2Protocol Off > AuthzSVNReposRelativeAccessFile authz > Requests to that location are properly forwarded to mod_dav_svn, but that fails to calculate the ultimate repo to use: > (2)No such file or directory: [...] Failed to load the mod_authz_svn config: > Can't open file '/home/ams_svn_repos/Bin/LSG BE-BB/svn/conf/authz': No such > file or directory > Access denied: - OPTIONS svn:/Bin/LSG BE-BB/GosaPrint/trunk When I change the location to use an underscore and request that, things succeed: > > Access denied: - OPTIONS GosaPrint:/trunk "Access denied" is the correct thing to do in this case, the important point is that the correct authz-file has been used, the correct repo extracted from the URL etc. So the problem is obviously with the space, but when I change the "Location" of httpd to contain "%20", that doesn't work, because httpd doesn't match the location at all anymore: > > Fehler: The server at > Fehler: '[...]/svn/Bin/LSG%20BE-BB/GosaPrint/trunk' > Fehler: does not support the HTTP/DAV protocol If I provide other locations, those get matched instead, like "/svn/Bin" and mod_dav_svn calculates another wrong repo name. So it's not that things don't work anymore at all, but only that httpd seems to really expect a space in the "Location". And I think that's the problem: The matched location is forwarded using a space to mod_dav_svn, which tries to calculate the repo name from the request and that request most likely containes %20 instead of the space. So calculation simply fails: > /* The URL space defined by the SVN provider is always a virtual > space. Construct the path relative to the configured Location > (root_path). So... the relative location is simply the URL used, > skipping the root_path. > Note: mod_dav has canonialized root_path. It will not have a trailing > slash (unless it is "/"). > Note: given a URI of /something and a root of /some, then it is >impossible to be here (and end up with "thing"). This is simply >because we control /some and are dispatched to here for its >URIs. We do not control /something, so we don't get here. Or, >if we *do* control /something, then it is for THAT root. > */ > relative = ap_stripprefix(uri, root_path); https://github.com/apache/subversion/blob/trunk/subversion/mod_dav_svn/repos.c#L1326 In the above code, "uri" and "root_path" would need to have the same representation, either both being encoded or decoded, but I guess both are different. Because of that, the "root_path" is not stripped, which leads to "svn" from the URL being the first component and hence use wrongly as repo name to map into "SVNParentPath". Similar problems with difference in "Location[Match]" vs. URLs of the request have been discussed in the past already: https://bz.apache.org/bugzilla/show_bug.cgi?id=35077 https://svn.haxx.se/dev/archive-2005-09/0470.shtml But I couldn't find anything simple like using spaces in URLs to repos. So, does that make sense to you? Has that problem already been reported in the past and I simply didn't find it? Is this worth reporting a bug or already fixed in current versions of SVN? Any workarounds you can think of with being able to keep the space in the URL? Thanks! Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Segfault in svnserve on UB 16.04 LTS sometimes, possible use-after-free?
Guten Tag Stefan Sperling, am Donnerstag, 18. April 2019 um 23:39 schrieben Sie: > You should upgrade. And please ask Ubuntu to stop shipping outdated > software with known bugs ;-) Thanks. Will do, even UB 18.04 LTS ships SVN 1.9.7 only and I don't see your mentioned patch backported. Thanks for the hint! Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Segfault in svnserve on UB 16.04 LTS sometimes, possible use-after-free?
ing the problem to the user mailing list, I was instructed
to install debug symbols and get a core dump with some stacktrace and
did so.
https://lists.apache.org/thread.html/d51cd58613c1033a0e53210346f901cb264d7da77723e4a80412ebc8@
> Stacktrace:
> #0 object_ref_cleanup (baton=0x7f5f75994f00) at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/libsvn_subr/object_pool.c:148
> object = 0x7f5f75994f00
> object_pool =
> #1 0x7f5f747e4e3e in apr_pool_destroy () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> No symbol table info available.
> #2 0x7f5f747e4e15 in apr_pool_destroy () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> No symbol table info available.
> #3 0x00406e4e in main (argc=, argv=0x7ffddc135568)
> at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/svnserve/svnserve.c:1368
> pool = 0x7f5f759d6028
> exit_code = 0
> err =
> StacktraceAddressSignature:
> /usr/bin/svnserve:11:/usr/lib/x86_64-linux-gnu/libapr-1.so.0.5.2+1ae3e:/usr/lib/x86_64-linux-gnu/libapr-1.so.0.5.2+1ae15:/usr/bin/svnserve+6e4e
> StacktraceSource:
> #0 object_ref_cleanup (baton=0x7f5f75994f00) at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/libsvn_subr/object_pool.c:148
>143: */
>144: static apr_status_t
>145: object_ref_cleanup(void *baton)
>146: {
>147: object_ref_t *object = baton;
>148: svn_object_pool__t *object_pool = object->object_pool;
>149:
>150: /* If we released the last reference to object, there is one more
>151: unused entry.
>152:
>153: Note that unused_count does not need to be always exact but only
> #1 0x7f5f747e4e3e in apr_pool_destroy () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> #2 0x7f5f747e4e15 in apr_pool_destroy () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> #3 0x00406e4e in main (argc=, argv=0x7ffddc135568)
> at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/svnserve/svnserve.c:1368
>1363: if (threads)
>1364: apr_thread_pool_destroy(threads);
>1365: #endif
>1366:
>1367: /* this will also close the server's socket */
>1368: svn_pool_destroy(pool);
>1369: return exit_code;
>1370: }
>
> StacktraceTop:
> object_ref_cleanup (baton=0x7f5f75994f00) at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/libsvn_subr/object_pool.c:148
> apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> main (argc=, argv=0x7ffddc135568) at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/svnserve/svnserve.c:1368
> ThreadStacktrace:
> .
> Thread 1 (Thread 0x7f5f759c9780 (LWP 3769)):
> #0 object_ref_cleanup (baton=0x7f5f75994f00) at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/libsvn_subr/object_pool.c:148
> object = 0x7f5f75994f00
> object_pool =
> #1 0x7f5f747e4e3e in apr_pool_destroy () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> No symbol table info available.
> #2 0x7f5f747e4e15 in apr_pool_destroy () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> No symbol table info available.
> #3 0x00406e4e in main (argc=, argv=0x7ffddc135568)
> at
> /build/subversion-8E3yhQ/subversion-1.9.3/subversion/svnserve/svnserve.c:1368
> pool = 0x7f5f759d6028
> exit_code = 0
> err =
Does the above help? I've attched some more verbose chrashlog but
without the actual dump, as that is some MiB in size. I can upload
that somewhere if it's considered necessary. Still don't know which
repo and operation triggers the problem.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail: thorsten.schoen...@am-soft.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...05151- 9468- 55
Fax...05151- 9468- 88
Mobil..0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
_usr_bin_svnserve.1207.crash w_o dump
Description: Binary data
