Bug#760841: pspp: [kfreebsd] random testsuite failures
Package: pspp Version: 0.8.3-4 Severity: important User: debian-...@lists.debian.org Usertags: kfreebsd X-Debbugs-CC: pspp-...@gnu.org, debian-...@lists.debian.org Hi, https://buildd.debian.org/status/fetch.php?pkg=pspp&arch=kfreebsd-amd64&ver=0.8.3-4&stamp=1408700427 On 08/09/14 10:03, Friedrich Beckmann wrote: > The package does not build because one test (read UTF-16 as Auto) in the > regression fails. The strange thing is > that the test already passed in a previous build attempt for package > 0.8.3-2 but there is no difference in the source code. > Even stranger is that the package failed on kfreebsd-i386 for version > 0.8.3-1.3 with exactly the same failing test. I'm opening a bug to track this issue because, even though it was given back and built successfully, it may recur in future, and it suggests a bug somewhere that we'd like to fix. > Ben Pfaff could not reproduce the problem on machine > fischer: http://lists.gnu.org/archive/html/pspp-dev/2014-08/msg00025.html > and so we left it and it build for 0.8.3-2 with no source code change on > kfreebsd-i386. I couldn't reproduce it yet either. But I think we have a race condition somewhere, perhaps in pspp, another library, the test, or even the test framework. I'll note that on the kfreebsd buildds the testsuite runs with -j2 Thanks, Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#760841: pspp: [kfreebsd] random testsuite failures
Hi again,
The failing test ("test 940", but that number may change) is this:
> AT_SETUP([read UTF-16 as Auto])
> AT_KEYWORDS([u8_istream])
> AT_CHECK([i18n-test supports_encodings UTF-16 UTF-16BE UTF-16LE])
This check seems fine:
> AT_CHECK([printf '\0e\0n\0t\0r\0\351\0e\0\n' | u8-istream-test read - Auto],
> [0], [entrée
> ])
This is the check that is usually seen to fail (u8-istream.at:146) :
> AT_CHECK([printf 'e\0n\0t\0r\0\351\0e\0\n\0' | u8-istream-test read - Auto],
> [0], [entrée
> ])
It is supposed to decode the UTF-16LE string to UTF-8. The following
method reproduced it 1.9% of the time on my kfreebsd-amd64 system. When
it fails, it outputs the raw input string not converted at all:
> ~/pspp-0.8.3$ for i in $(seq 1 1000) ; do printf 'e\0n\0t\0r\0\351\0e\0\n\0'
> | tests/libpspp/u8-istream-test read - Auto | xxd ; done | sort | uniq -c |
> sort -bn
> 19 000: 6500 6e00 7400 7200 c3a9 0065 000a 00e.n.t.re...
> 981 000: 656e 7472 c3a9 650a entr..e.
This check seems fine:
> AT_CHECK([printf '\376\377\0e\0n\0t\0r\0\351\0e\0\n' | u8-istream-test read -
> Auto],
> [0], [entrée
> ])
This check seems to have issues too (UTF-16LE with Byte-Order Marker):
> AT_CHECK([printf '\377\376e\0n\0t\0r\0\351\0e\0\n\0' | u8-istream-test read -
> Auto],
> [0], [entrée
> ])
> ~/pspp-0.8.3$ for i in $(seq 1 1000) ; do printf
> '\377\376e\0n\0t\0r\0\351\0e\0\n\0' | tests/libpspp/u8-istream-test read -
> Auto | xxd ; done | sort | uniq -c | sort -bn
Raw input string output about 1.4% of the time:
> 14 000: c3bf c3be 6500 6e00 7400 7200 c3a9 0065 e.n.t.re
> 14 010: 000a 00 ...
Correct output:
> 986 000: 656e 7472 c3a9 650a entr..e.
Regards,
--
Steven Chamberlain
ste...@pyro.eu.org
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#760841: pspp: [kfreebsd] random testsuite failures
Thank you for trying the build, adding the bug description to the bugtracker and analyzing the problem! https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760841 Friedrich -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#760841: pspp: [kfreebsd] random testsuite failures
With some difficulty, I managed to reproduce the failure situation under ktrace, so that I could diff with the successful case: > u8-istream-test CALL read(0,0x640670,0x1000) > -u8-istream-test GIO fd 0 read 13 bytes > -0x 6500 6e00 7400 7200 e900 6500 0a > +u8-istream-test GIO fd 0 read 14 bytes > +0x 6500 6e00 7400 7200 e900 6500 0a00 > > -u8-istream-test RET read 13/0xd > +u8-istream-test RET read 14/0xe The size of the read from fd 0 (stdin) was different; the last null byte got truncated. I don't know why that is. The read is assumed to be complete and it doesn't attempt to read again. The effect is it will later decide the string is not valid UTF-16LE, so treats it as CP1252, and doesn't properly convert it. It seems logical this bug only happened for Little Endian encodings (strings with Big Endian encoding will not end with a null byte). > u8-istream-test CALL open(0x8012baba0,0,0) > u8-istream-test NAMI > "/usr/lib/x86_64-kfreebsd-gnu/gconv/gconv-modules.cache" > u8-istream-test RET open 3 > -u8-istream-test CALL fstat(0x3,0x7fffcf40) > +u8-istream-test CALL fstat(0x3,0x7fffd190) > u8-istream-test STRU invalid record > u8-istream-test RET fstat 0 > u8-istream-test CALL mmap(0,0x6696,0x1,0x1,0x3,0) > u8-istream-test RET mmap 34368200704/0x800812000 > u8-istream-test CALL close(0x3) > u8-istream-test RET close 0 > -u8-istream-test CALL open(0x6499c0,0x10,0) > -u8-istream-test NAMI "/usr/lib/x86_64-kfreebsd-gnu/gconv/CP1252.so" > +u8-istream-test CALL open(0x6417d0,0x10,0) > +u8-istream-test NAMI "/usr/lib/x86_64-kfreebsd-gnu/gconv/UTF-16.so" Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
