Control: tags 927027 + patch
Dear Maintainer, I tried to have a look at this crash and I think it is related to the large file support, which is defined in dcfldd.h, line 27 and 28. Unfortunately this file gets not included first in split.c and therefore off_t gets defined without large file support. Therefore the size of struct split_t in split.c and output.c is different and therefore the fmt is a null pointer. The include in split.c got introduced in 30_fix-implicit-declarations.patch and might be fixed there. There is another case where dcfldd.h is not the first include in file verify.c, but cannot say if that would cause other problems. Attached patch demonstrates both changes and a package built with it could run the test successful. I have just tested i386, not armhf. Kind regards, Bernhard (gdb) bt #0 __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:50 #1 0x0041da62 in getext (num=0, fmt=0x0) at split.c:52 #2 open_split (split=0x2262330) at split.c:104 #3 split_write (split=0x2262330, buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=<optimized out>) at split.c:128 #4 0x0041d96d in outputlist_write (buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=18171) at output.c:155 #5 0x0041b6a7 in dd_copy () at copy.c:366 #6 0x00414d29 in main (argc=<optimized out>, argv=<optimized out>) at dcfldd.c:768
# Buster i386 qemu VM 2019-04-14 apt update apt dist-upgrade apt install dpkg-dev devscripts mc systemd-coredump autopkgtest dcfldd dcfldd-dbgsym gdb lz4 apt build-dep dcfldd mkdir /home/benutzer/source/dcfldd/orig -p cd /home/benutzer/source/dcfldd/orig apt source dcfldd cd cd /home/benutzer/source/dcfldd cp -a orig try1 cd try1/dcfldd-1.3.4.1 dpkg-buildpackage autopkgtest -- null ########### benutzer@debian:~/source/dcfldd/try1/dcfldd-1.3.4.1$ autopkgtest -- null autopkgtest [18:54:30]: version 5.10 autopkgtest [18:54:30]: host debian; command line: /usr/bin/autopkgtest -- null autopkgtest [18:54:30]: testbed dpkg architecture: i386 autopkgtest [18:54:30]: testbed running kernel: Linux 4.19.0-4-686-pae #1 SMP Debian 4.19.28-2 (2019-03-15) autopkgtest [18:54:30]: @@@@@@@@@@@@@@@@@@@@ built-tree . autopkgtest [18:54:30]: testing package dcfldd version 1.3.4.1-13 autopkgtest [18:54:30]: test command1: preparing testbed Starting pkgProblemResolver with broken count: 0 Starting 2 pkgProblemResolver with broken count: 0 Done autopkgtest: WARNING: virtualisation system does not offer root or writable testbed; unpacking dependencies to temporary dir, which will only work for some packages Holen:1 http://192.168.178.25:9999/debian-10-buster-deb.debian.org buster/main i386 dcfldd i386 1.3.4.1-13 [42,5 kB] Es wurden 42,5 kB in 0 s geholt (2.439 kB/s). autopkgtest [18:54:31]: test command1: dcfldd if=debian/tests/test.txt split=1000 of=$AUTOPKGTEST_TMP/test2.txt hash=md5,sha1,sha256,sha384,sha512; ls $AUTOPKGTEST_TMP -lh autopkgtest [18:54:31]: test command1: [----------------------- bash: line 1: 19173 Segmentation fault (core dumped) dcfldd if=debian/tests/test.txt split=1000 of=$AUTOPKGTEST_TMP/test2.txt hash=md5,sha1,sha256,sha384,sha512 autopkgtest [18:54:32]: test command1: -----------------------] autopkgtest [18:54:32]: test command1: - - - - - - - - - - results - - - - - - - - - - command1 FAIL non-zero exit status 139 autopkgtest [18:54:32]: @@@@@@@@@@@@@@@@@@@@ summary command1 FAIL non-zero exit status 139 ########### dmesg: [ 744.195515] dcfldd[19173]: segfault at 0 ip b7e38686 sp bff82094 error 4 in libc-2.28.so[b7dca000+14e000] [ 744.195524] Code: 89 fa 5b 5e 0f b6 fa 29 f8 5f 5d c3 66 90 66 90 66 90 66 90 56 57 8b 7c 24 0c 31 c0 89 f9 83 e1 3f 66 0f ef c0 83 f9 30 77 17 <f3> 0f 6f 0f 66 0f 74 c1 66 0f d7 d0 85 d2 75 73 89 f8 83 e0 f0 eb root@debian:~# coredumpctl list TIME PID UID GID SIG COREFILE EXE Sun 2019-04-14 18:54:31 CEST 19173 1000 1000 11 present /tmp/autopkgtest.eHCLuf/deps/usr/bin/dcfldd root@debian:~# coredumpctl gdb 19173 PID: 19173 (dcfldd) UID: 1000 (benutzer) GID: 1000 (benutzer) Signal: 11 (SEGV) Timestamp: Sun 2019-04-14 18:54:31 CEST (1min 5s ago) Command Line: dcfldd if debian/tests/test.txt split 1000 of /tmp/autopkgtest.eHCLuf/autopkgtest_tmp/test2.txt hash md5 sha1 sha256 sha384 sha512 Executable: /tmp/autopkgtest.eHCLuf/deps/usr/bin/dcfldd Control Group: /user.slice/user-1000.slice/session-3.scope Unit: session-3.scope Slice: user-1000.slice Session: 3 Owner UID: 1000 (benutzer) Boot ID: 8b4e680d36e04a738bc8e8a33f18f3c0 Machine ID: 45f49504b47f4e5690bc479adf67aa5b Hostname: debian Storage: /var/lib/systemd/coredump/core.dcfldd.1000.8b4e680d36e04a738bc8e8a33f18f3c0.19173.1555260871000000.lz4 Message: Process 19173 (dcfldd) of user 1000 dumped core. Stack trace of thread 19173: #0 0x00000000b7e38686 __strlen_sse2_bsf (libc.so.6) #1 0x000000000041da62 n/a (/tmp/autopkgtest.eHCLuf/deps/usr/bin/dcfldd) GNU gdb (Debian 8.2.1-2) 8.2.1 Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... /tmp/autopkgtest.eHCLuf/deps/usr/bin/dcfldd: Datei oder Verzeichnis nicht gefunden. [New LWP 19173] Core was generated by `dcfldd if debian/tests/test.txt split 1000 of /tmp/autopkgtest.eHCLuf/autopkgte'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0xb7e38686 in ?? () (gdb) set width 0 (gdb) set pagination off (gdb) bt #0 0xb7e38686 in ?? () #1 0x00426ea4 in ?? () Backtrace stopped: previous frame inner to this frame (corrupt stack?) ######## cp -a /var/lib/systemd/coredump/core.dcfldd.1000.8b4e680d36e04a738bc8e8a33f18f3c0.19173.1555260871000000.lz4 /tmp/ unlz4 /tmp/core.dcfldd.1000.8b4e680d36e04a738bc8e8a33f18f3c0.19173.1555260871000000.lz4 root@debian:~# gdb -q /usr/bin/dcfldd --core /tmp/core.dcfldd.1000.8b4e680d36e04a738bc8e8a33f18f3c0.19173.1555260871000000 Reading symbols from /usr/bin/dcfldd...(no debugging symbols found)...done. [New LWP 19173] Core was generated by `dcfldd if debian/tests/test.txt split 1000 of /tmp/autopkgtest.eHCLuf/autopkgte'. Program terminated with signal SIGSEGV, Segmentation fault. #0 __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:50 50 ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S: Datei oder Verzeichnis nicht gefunden. (gdb) set width 0 (gdb) set pagination off (gdb) bt #0 __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:50 #1 0x0041da62 in ?? () #2 0x0041d96d in ?? () #3 0x0041b6a7 in ?? () #4 0x00414d29 in ?? () #5 0xb7dcbb41 in __libc_start_main (main=0x414480, argc=5, argv=0xbff82344, init=0x41e680, fini=0x41e6e0, rtld_fini=0xb7fac560 <_dl_fini>, stack_end=0xbff8233c) at ../csu/libc-start.c:308 #6 0x00415469 in ?? () With debug symbols: root@debian:~# gdb -q /usr/bin/dcfldd --core /tmp/core.dcfldd.1000.8b4e680d36e04a738bc8e8a33f18f3c0.19173.1555260871000000 Reading symbols from /usr/bin/dcfldd...Reading symbols from /usr/lib/debug/.build-id/29/b8e41c09e64820ca2a92f49931609780a75c81.debug...done. done. [New LWP 19173] Core was generated by `dcfldd if debian/tests/test.txt split 1000 of /tmp/autopkgtest.eHCLuf/autopkgte'. Program terminated with signal SIGSEGV, Segmentation fault. #0 __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:50 50 ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S: Datei oder Verzeichnis nicht gefunden. (gdb) set width 0 (gdb) set pagination off (gdb) directory /home/benutzer/source/dcfldd/try1/dcfldd-1.3.4.1 Source directories searched: /home/benutzer/source/dcfldd/try1/dcfldd-1.3.4.1:$cdir:$cwd (gdb) bt #0 __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:50 #1 0x0041da62 in getext (num=0, fmt=0x0) at split.c:52 #2 open_split (split=0x2262330) at split.c:104 #3 split_write (split=0x2262330, buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=<optimized out>) at split.c:128 #4 0x0041d96d in outputlist_write (buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=18171) at output.c:155 #5 0x0041b6a7 in dd_copy () at copy.c:366 #6 0x00414d29 in main (argc=<optimized out>, argv=<optimized out>) at dcfldd.c:768 (gdb) up #1 0x0041da62 in getext (num=0, fmt=0x0) at split.c:52 warning: Source file is more recent than executable. 52 int fmtlen = strlen(fmt); (gdb) list 47 /* Generate a split file extension string based on 48 * the specified format string and a given number 49 */ 50 static char *getext(char *fmt, int num) 51 { 52 int fmtlen = strlen(fmt); <<<<<<<<<<<<<<<<< 53 int i; 54 char *retval; 55 56 assert(fmtlen > 0); 57 58 retval = malloc(fmtlen); 59 60 /* Fill the retval in reverse while constantly dividing num apropriately */ 61 for (i = fmtlen - 1; i >= 0; i--) { 62 int x; 63 64 if (fmt[i] == 'a') { 65 x = num % NUM_LETTERS; 66 retval[i] = letters[x]; 67 num = num / NUM_LETTERS; 68 } else { 69 x = num % NUM_NUMBERS; 70 retval[i] = numbers[x]; 71 num = num / NUM_NUMBERS; 72 } 73 } 74 75 retval[fmtlen] = '\0'; 76 77 return retval; 78 } (gdb) up #2 open_split (split=0x2262330) at split.c:104 104 ext = getext(split->format, splitnum); (gdb) list split.c:96,120 96 /* Open the next extension in a split sequence */ 97 static void open_split(split_t *split) 98 { 99 int fd; 100 int splitnum = split->total_bytes / split->max_bytes; 101 mode_t perms = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH; 102 char *ext, *fname; 103 104 ext = getext(split->format, splitnum); <<<<<<<<<<<<<<<<< 105 /* [FIX] split.c:105:5: warning: ignoring return value of ‘asprintf’, declared with attribute warn_unused_result [-Wunused-result] */ 106 if( asprintf(&fname, "%s.%s", split->name, ext) == -1) { 107 return; 108 } 109 free(ext); 110 111 fd = open(fname, O_WRONLY | O_CREAT, perms); 112 113 if (fd < 0) 114 syscall_error(fname); 115 116 split->currfd = fd; 117 split->curr_bytes = 0; 118 119 free(fname); 120 } (gdb) print split->format $1 = 0x0 (gdb) print split $2 = (split_t *) 0x2262330 (gdb) print *split $3 = {currfd = -1, max_bytes = 1000, total_bytes = 0, curr_bytes = 0, name = 0x0, format = 0x0} (gdb) up #3 split_write (split=0x2262330, buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=<optimized out>) at split.c:128 128 open_split(split); (gdb) list split.c:121,150 121 122 int split_write(split_t *split, const char *buf, size_t len) 123 { 124 off_t left = split->max_bytes - split->curr_bytes; 125 int nwritten = 0; 126 127 if (left == 0 || split->currfd == -1) { 128 open_split(split); <<<<<<<<<<<<<<<<< 129 left = split->max_bytes; 130 } 131 132 if (len <= left) { 133 nwritten = full_write(split->currfd, buf, len); 134 split->total_bytes += nwritten; 135 split->curr_bytes += nwritten; 136 } else { 137 nwritten = full_write(split->currfd, buf, left); 138 split->total_bytes += nwritten; 139 split->curr_bytes += nwritten; 140 nwritten += split_write(split, &buf[nwritten], len - nwritten); 141 } 142 143 return nwritten; 144 } (gdb) print *split $4 = {currfd = -1, max_bytes = 1000, total_bytes = 0, curr_bytes = 0, name = 0x0, format = 0x0} (gdb) print sizeof(*split) $8 = 24 (gdb) print &split->format $9 = (char **) 0x2262344 (gdb) up #4 0x0041d96d in outputlist_write (buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=18171) at output.c:155 155 nwritten = split_write(ptr->data.split, buf, len); (gdb) list output.c:143,170 143 int outputlist_write(const char *buf, size_t len) 144 { 145 outputlist_t *ptr; 146 int nwritten = 0; 147 148 for (ptr = outputlist; ptr != NULL; ptr = ptr->next) { 149 nwritten = 0; 150 switch (ptr->type) { 151 case SINGLE_FILE: 152 nwritten = full_write(ptr->data.fd, buf, len); 153 break; 154 case SPLIT_FILE: 155 nwritten = split_write(ptr->data.split, buf, len); <<<<<<<<<<<<<<<<< 156 break; 157 } 158 if (nwritten < len) 159 break; 160 } 161 162 return nwritten; 163 } (gdb) print ptr->data.split $5 = (split_t *) 0x2262330 (gdb) print *ptr->data.split $6 = {currfd = -1, max_bytes = 1000, total_bytes = 0, curr_bytes = 0, name = 0x2262360 "/tmp/autopkgtest.eHCLuf/autopkgtest_tmp/test2.txt", format = 0x22623a0 "nnn"} (gdb) print sizeof(*ptr->data.split) $7 = 36 (gdb) print &ptr->data.split->format $10 = (char **) 0x2262350 ########## (gdb) down #3 split_write (split=0x2262330, buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=<optimized out>) at split.c:128 128 open_split(split); (gdb) set print pretty on (gdb) print split $21 = (split_t *) 0x2262330 (gdb) print *split $12 = { currfd = -1, max_bytes = 1000, total_bytes = 0, curr_bytes = 0, name = 0x0, format = 0x0 } (gdb) print sizeof(split->currfd) $13 = 4 (gdb) print sizeof(split->max_bytes) $14 = 4 (gdb) print sizeof(split->total_bytes) $15 = 4 (gdb) print sizeof(split->curr_bytes) $16 = 4 (gdb) print sizeof(split->name) $17 = 4 (gdb) print sizeof(split->format) $18 = 4 (gdb) up #4 0x0041d96d in outputlist_write (buf=0x226d000 "Autopkgtest - Defining tests for Debian packages\n\nThis document describes how autopkgtest interprets and executes tests found in Debian source packages.\nOverview\n\nThe source package provides a test me"..., len=18171) at output.c:155 155 nwritten = split_write(ptr->data.split, buf, len); (gdb) print ptr->data.split $20 = (split_t *) 0x2262330 (gdb) print *ptr->data.split $19 = { currfd = -1, max_bytes = 1000, total_bytes = 0, curr_bytes = 0, name = 0x2262360 "/tmp/autopkgtest.eHCLuf/autopkgtest_tmp/test2.txt", format = 0x22623a0 "nnn" } (gdb) print sizeof(ptr->data.split->currfd) $22 = 4 (gdb) print sizeof(ptr->data.split->max_bytes) $23 = 8 (gdb) print sizeof(ptr->data.split->total_bytes) $24 = 8 (gdb) print sizeof(ptr->data.split->curr_bytes) $25 = 8 (gdb) print sizeof(ptr->data.split->name) $26 = 4 (gdb) print sizeof(ptr->data.split->format) $27 = 4 set width 0 set pagination off directory /home/benutzer/source/dcfldd/try1/dcfldd-1.3.4.1 bt ############## https://buildd.debian.org/status/fetch.php?pkg=dcfldd&arch=i386&ver=1.3.4.1-13&stamp=1543365026&raw=0 gcc -DHAVE_CONFIG_H -I. -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -c -o split.o split.c gcc -DHAVE_CONFIG_H -I. -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -c -o output.o output.c -> no difference in compiler command line ... -> maybe include order? benutzer@debian:~/source/dcfldd/try2/dcfldd-1.3.4.1$ grep LARGE dcfldd.h -n -B1 27-#define _FILE_OFFSET_BITS 64 28:#define LARGEFILE_SOURCE 30_fix-implicit-declarations.patch -> introduces the "#include <stdio.h>" before dcfldd.h in split.c ############## benutzer@debian:~/source/dcfldd/try1/dcfldd-1.3.4.1$ dcfldd if=debian/tests/test.txt split=1000 of=/tmp/test2.txt hash=md5,sha1,sha256,sha384,sha512 Speicherzugriffsfehler (Speicherabzug geschrieben) ############## cd /home/benutzer/source/dcfldd/ cp -a orig try2 cd try2/dcfldd-1.3.4.1 # changing files dpkg-source --commit # 80_fix-927027-include-order.patch dpkg-buildpackage benutzer@debian:~/source/dcfldd/try2/dcfldd-1.3.4.1$ ./dcfldd if=debian/tests/test.txt split=1000 of=/tmp/test2.txt hash=md5,sha1,sha256,sha384,sha512 Total (md5): 92994b0ce292a217e3e3bc31b639e565 Total (sha1): bc0e4b74695142e0a0bdae87aea310d7078866cb Total (sha256): 02fd428a4671925e4ca61541b9fac648f4ccdccad65602bfd3256ba14e59489c Total (sha384): 53b8374607a8258de4173265bfbfa6120093fd42090a92fd589cf2c6c16b4e421b5135777714976713f7949715720a83 Total (sha512): 969a39bf47b5f12d81121084f19cb5ae250e0c0ea5b7c6d82cf08131acde8e1955d137612c2f6b255e25b0e28f96f93586f90f06965cb8f719ed7fbdd95cc8d4 0+1 records in 0+1 records out #############
Description: Fix include order to avoid different definitions of off_t. Author: Bernhard Ãbelacker <bernha...@mailbox.org> Bug-Debian: https://bugs.debian.org/927027 Last-Update: 2019-04-14 --- dcfldd-1.3.4.1.orig/split.c +++ dcfldd-1.3.4.1/split.c @@ -20,10 +20,11 @@ /* GNU dd originally written by Paul Rubin, David MacKenzie, and Stuart Kemp. */ +#include "dcfldd.h" + #define _GNU_SOURCE 1 #include <stdio.h> -#include "dcfldd.h" #include "split.h" #include <stdlib.h> #include <string.h> --- dcfldd-1.3.4.1.orig/verify.c +++ dcfldd-1.3.4.1/verify.c @@ -21,11 +21,11 @@ /* GNU dd originally written by Paul Rubin, David MacKenzie, and Stuart Kemp. */ +#include "dcfldd.h" #include <stdlib.h> #include <sys/types.h> #include <string.h> #include "config.h" -#include "dcfldd.h" #include "hash.h" #include "getpagesize.h" #include "safe-read.h"
