On 14 Jul 2021, at 21:42, Nathan Hartman <hartman.nat...@gmail.com> wrote: > > On Tue, Jul 13, 2021 at 5:49 PM Attila <atiw...@gmx.net > <mailto:atiw...@gmx.net>> wrote: >> >> Hi >> >> I have a problem getting the svn log in a branch after sync-merging a commit >> from trunk. >> This commit in trunk is a merge of an old and complex branch with many >> commits. >> >> The client accessing the repository over svn:// url. >> (paths and text is redacted) >> The head revision is: 10801 >> >> When I run the following command on the client (in the working copy), it >> prints a long partial xml-log output, then hangs. >> /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc >> >> When observing in "top", the command uses no visible CPU resources on hang. >> (I waited ca. 2 minutes) >> The hanging command does mot exits on CTRL-c, it does not exits on "kill >> -TERM pid", I have to send "kill -KILL pid" to terminate it. >> >> When I run the command with strace it hangs at read(4, >> ...SNIP... >> read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d83000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d81000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d7f000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d7d000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d7b000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d79000 >> read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d77000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d75000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d73000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d71000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d6f000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7f63f1d6d000 >> read(4, >> >> When I observe the server, there is a CPU activity at the begin, but when >> the client hangs, the server seems to be in idle. >> Just a corresponding svnserve process is there with no visible cpu usage. >> In svnserve.log is nothing relevant to see. >> >> The svnserve command is: >> svnserve -d -r /path/to/repositories \ >> --log-file=/var/log/svnserve.log \ >> --memory-cache-size 1024 \ >> --cache-txdeltas yes \ >> --cache-fulltexts yes >> >> When I try to get the xml-log on the server with the corresponding file:// >> repository URL: >> /usr/bin/svn log --xml -g -v -r 10701:HEAD >> file://path/to/local/repositories/project/branch >> The command finishes in ca 5-10 seconds and I get the xml output, but the >> output has a way too many </logentry> lines. >> >> There are 1217 occurrences of the string “<logentry" and 33191 occurrences >> of the string "</logentry>" in the output xml. >> There are several thousand lines of </logentry> in a row in many places in >> repeated blocks. >> >> Details: >> Client and Server OS: >> Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 >> GNU/Linux >> >> The repository is ca. 4 GB. >> Running "svnadmin verify" on the server founds no errors. >> I have no other problems with the server, checkout and commit works normal. >> >> svn --version >> svn, version 1.10.4 (r1850624) >> compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu >> >> Copyright (C) 2019 The Apache Software Foundation. >> This software consists of contributions made by many people; >> see the NOTICE file for more information. >> Subversion is open source software, see http://subversion.apache.org/ >> >> The following repository access (RA) modules are available: >> >> * ra_svn : Module for accessing a repository using the svn network protocol. >> - with Cyrus SASL authentication >> - handles 'svn' scheme >> * ra_local : Module for accessing a repository on local disk. >> - handles 'file' scheme >> * ra_serf : Module for accessing a repository via WebDAV protocol using serf. >> - using serf 1.3.9 (compiled with 1.3.9) >> - handles 'http' scheme >> - handles 'https' scheme >> >> The following authentication credential caches are available: >> >> * Plaintext cache in /home/username/.subversion >> * Gnome Keyring >> * GPG-Agent >> * KWallet (KDE) > > > Hello Attila, > > Thanks for this detailed explanation. There are quite a few important > clues here. > > To help narrow down the search for the culprit: Are you able to run > the same 'svn log' command against the same working copy, but without > the '--xml', and get a correct output in a reasonable amount of time?
Hi Nathan thanks for the suggestions. The command without the --xml parameter hangs on the client and works on the server. The text output on the server seems be ok. This suggests that the hang and the closing xml tags are two unrelated problems. > > If log without '--xml' works and log with '--xml' gets stuck, is it > feasible to try to bisect the revision range, e.g., run log on the > range r10701:10751, and in a separate invocation on r10751:10801, and I tried that, it seems that the problem was not the last sync-merge commit. The command succeeds when there is no sync-merge commit in the range. There are two sync-merges in the range the first one is a small merge of 3-4 changed files. These 3-4 files are unmodified in the branch. Whenever i include one of these sync-merges in the range, the command hangs. > try to see if there's a revision in particular where SVN is getting > confused? If so, then perhaps knowing a little more about that > revision could help narrow it down. > > By the way, is this (1.10.4) the SVN version on the server, the > client, or both? > both. The server and client is Debian Buster and I have updated both to the latest version hoping it will help. The installed subversion package is: subversion/stable,stable,now 1.10.4-1+deb10u2 amd64 [installed] The client is a private vagrant vm running on macOS in VMware Fusion pro, the server is an ESXi vm. >> Googling around gives me these two somewhat related hits: >> https://issues.apache.org/jira/browse/SVN-4856 >> <https://issues.apache.org/jira/browse/SVN-4856> >> https://issues.apache.org/jira/browse/SVN-4711 >> <https://issues.apache.org/jira/browse/SVN-4711> >> >> But I do'nt use the --search parameter. > > It could nevertheless be related. The spurious XML closing tags seems > to ring a bell. > >> Is this a bug or are there any suggestions how to solve this problem? > > It sounds like a bug. My suggestions above. Here is a preliminary backtrace of the hanging client (without debug symbols for now) I need some time to spin up a copy of the server and the client and re-compile svn with debug symbols. I will come back with more details as soon as i have the time to do more tests. # gdb /usr/bin/svn 1399 GNU gdb (Debian 8.2.1-2+b3) 8.2.1 Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/bin/svn...(no debugging symbols found)...done. Attaching to program: /usr/bin/svn, process 1399 Reading symbols from /lib/x86_64-linux-gnu/libsvn_client-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_wc-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_delta-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_diff-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_subr-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libaprutil-1.so.0...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libapr-1.so.0...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_repos-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_serf-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsqlite3.so.0...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libutf8proc.so.2...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols from /usr/lib/debug/.build-id/c6/966ba263bacba7f67c23643ba373aa51a162e9.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done. done. Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_fs-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_x-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_base-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_util-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libserf-1.so.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libdb-5.3.so...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols from /usr/lib/debug/.build-id/02/6c3ba167f64f631eb8781fca2269fbc2ee7ca5.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.1...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libk5crypto.so.3...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libkrb5support.so.0...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so...(no debugging symbols found)...done. Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libplain.so...(no debugging symbols found)...done. Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols from /usr/lib/debug/.build-id/4b/ff8b782e1602c596e856bdef06e642e50e7fa7.debug...done. done. Reading symbols from /lib/x86_64-linux-gnu/libnss_dns.so.2...Reading symbols from /usr/lib/debug/.build-id/bb/8b756d3859e8fe20d7aa6d8255f27942589f47.debug...done. done. 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26 26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory. (gdb) bt #0 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26 #1 0x00007f276a22f37f in apr_socket_recv () from /lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x00007f2769fc1353 in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #3 0x00007f2769fc158c in svn_ra_svn.stream_read () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #4 0x00007f2769fbb242 in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #5 0x00007f2769fbb61a in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #6 0x00007f2769fbb68b in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #7 0x00007f2769fbbc7f in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #8 0x00007f2769fbbcbe in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #9 0x00007f2769fbde67 in svn_ra_svn.read_item () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #10 0x00007f2769fb583a in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1 #11 0x00007f276a46033a in svn_client_log5 () from /lib/x86_64-linux-gnu/libsvn_client-1.so.1 #12 0x000055f9507b6779 in ?? () #13 0x000055f9507c2541 in ?? () #14 0x000055f9507a7513 in ?? () #15 0x00007f276a04e09b in __libc_start_main (main=0x55f9507a7490, argc=7, argv=0x7ffee0fc23c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffee0fc23b8) at ../csu/libc-start.c:308 #16 0x000055f9507a759a in ?? () (gdb) Thanks Attila
