[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[ https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719457#comment-16719457 ] Roger Leigh commented on XERCESC-2162: -- I'm unsure why volatile would be a needed qualifier on these specific variables. They shouldn't be optimised out, irrespective of whether volatile is added or not. fHeartBeat is intrinsically racy from what I can see; it's modified by both the main thread and the corresponding worker, and read in the main thread, but it's only used for some cosmetic reporting symbols as far as I can see. Moreover, we always run the thread tests with -quiet, so "if (gRunInfo.quiet == false..." on 1306 should always be false. fHeartBeat should be written once by the worker for each pass and never read. fInProgress is written in the worker and checked in the main thread at termination; should be safe given how it's used with or without volatile. I can see this deadlocking if a worker terminates early; it will never be reset back to false. fParses isn't actually used except for some trivial reporting; shouldn't be optimised out though fThreadNum is set once in the main thread and read by the worker, should be safe Are all four of these structure members definitely optimised out? Can you put a breakpoint in one of the workers and check each member at e.g. thInfo at line 1077 and 1150 for a few passes. I'm suspicious that the volatile qualifiers are hiding some deeper problem. It shouldn't be required for multi-threaded code, and neither GCC, Clang nor MSVC are optimising anything away. The compiler doesn't have sufficient information to elide these members as far as I can see. They are being passed by pointer to the thread main for each worker, and that should surely be a barrier to optimisation; the compiler should not be able to determine that it's not modified at this point, and that should kill any elision. At least, that's my take on it, but I could certainly be wrong. The 21 second timeout is because the test is run with "-time 20" and there's a final 1 second delay while the threads terminate. This is adjustable in the test configuration. > ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163 > --- > > Key: XERCESC-2162 > URL: https://issues.apache.org/jira/browse/XERCESC-2162 > Project: Xerces-C++ > Issue Type: Bug > Components: Samples/Tests >Affects Versions: 3.2.2 > Environment: cat /etc/redhat-release > Red Hat Enterprise Linux Server release 7.6 (Maipo) > uname -a > Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux > Two versions of the Intel compiler suite were tried: > icpc --version > icpc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 17.0.5 20170817 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icpc --version > icpc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. > icc --version > icc (ICC) 18.0.1 20171018 > Copyright (C) 1985-2017 Intel Corporation. All rights reserved. >Reporter: Sam Trahan >Priority: Major > > The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel > compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in > gdb reveals that all ten threads exit, and main() is stuck in a wait loop > calling sleep() forever. > export CXX=icpc > export CFLAGS='-fp-model precise' > export CXXFLAGS='-fp-model precise' > export CC=icc > export CPP="icc -E" > export CXXCPP="icpc -E" > ./configure --prefix=/some/path > make > make check > Changing the CXXFLAGS to this does not help: > export CXXFLAGS='-fp-model precise -std=c++11' > The last bit of output from "make check:" > make[3]: Entering directory `/a-very-long-path/.../tests' > PASS: scripts/DOMTest > PASS: scripts/DOMMemTest > PASS: scripts/RangeTest > PASS: scripts/DOMTraversalTest > XFAIL: scripts/XSerializerTest > PASS: scripts/XSerializerTest1 > PASS: scripts/XSerializerTest2 > PASS: scripts/XSerializerTest3 > PASS: scripts/XSerializerTest4 > PASS: scripts/XSerializerTest5 > PASS: scripts/XSValueTest > XFAIL: scripts/InitTermTest > PASS: scripts/InitTermTest1 > PASS: scripts/InitTermTest2 > PASS: scripts/InitTermTest3 > XFAIL: scripts/ThreadTest > The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is > running: > /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet > -threads 10 -time 20 personal.xml -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org For additional commands, e-mail: c-dev-h...@xerces.apache.org
[jira] [Comment Edited] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[
https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719074#comment-16719074
]
Sam Trahan edited comment on XERCESC-2162 at 12/12/18 5:29 PM:
---
I tried three tests and determined the problem was a few missing "volatile"
declarations. The test takes 21.05 seconds to pass, which suggests to me that
there is another problem somewhere.
Test 1: no optimization - works
Test 2: full debugging plus -O3 - fails just the same
Test 3: add "volatile" declarations in four places, full debugging, plus -O3 -
works, but takes 21.05 seconds
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
The fHeartBeat is optimized away because it was not declared volatile:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
If I add "volatile" to these declarations, the thread tests pass, but take
21.05 seconds each to do it, which suggests there is another problem somewhere.
volatile boolfHeartBeat; // Set true by the thread each time
it finishes
// parsing a file.
volatile boolfInProgress; // Set to false by the thread when
parse in progress
volatile unsigned intfParses; // Number of parses completed.
volatile int fThreadNum; // Identifying number for this
thread.
was (Author: samueltrahannoaa):
I tried two tests and determined the problem was a few missing "volatile"
declarations. The test takes 21.05 seconds to pass, which suggests to me that
there is another problem somewhere.
Test 1: no optimization - works
Test 2: full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
The fHeartBeat is optimized away because it was not declared volatile:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
If I add "volatile" to these declarations, the thread tests pass, but take
21.05 seconds each to do it, which suggests there is another problem somewhere.
volatile boolfHeartBeat; // Set true by the thread each time
it finishes
// parsing a file.
volatile boolfInProgress; // Set to false by the thread when
parse in progress
volatile unsigned intfParses; // Number of parses completed.
volatile int fThreadNum; // Identifying number for this
thread.
> ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
> ---
>
> Key: XERCESC-2162
> URL: https://issues.apache.org/jira/browse/XERCESC-2162
> Project: Xerces-C++
> Issue Type: Bug
> Components: Samples/Tests
>Affects Versions: 3.2.2
> Environment: cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.6 (Maipo)
>
[jira] [Comment Edited] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[
https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719074#comment-16719074
]
Sam Trahan edited comment on XERCESC-2162 at 12/12/18 3:33 PM:
---
I tried two tests and determined the problem was a few missing "volatile"
declarations. The test takes 21.05 seconds to pass, which suggests to me that
there is another problem somewhere.
Test 1: no optimization - works
Test 2: full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
The fHeartBeat is optimized away because it was not declared volatile:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
If I add "volatile" to these declarations, the thread tests pass, but take
21.05 seconds each to do it, which suggests there is another problem somewhere.
volatile boolfHeartBeat; // Set true by the thread each time
it finishes
// parsing a file.
volatile boolfInProgress; // Set to false by the thread when
parse in progress
volatile unsigned intfParses; // Number of parses completed.
volatile int fThreadNum; // Identifying number for this
thread.
was (Author: samueltrahannoaa):
I tried two tests:
1. no optimization - works
2. full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
The fHeartBeat is optimized away, possibly because it was not declared volatile:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
> ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
> ---
>
> Key: XERCESC-2162
> URL: https://issues.apache.org/jira/browse/XERCESC-2162
> Project: Xerces-C++
> Issue Type: Bug
> Components: Samples/Tests
>Affects Versions: 3.2.2
> Environment: cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.6 (Maipo)
> uname -a
> Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64
> x86_64 x86_64 GNU/Linux
> Two versions of the Intel compiler suite were tried:
> icpc --version
> icpc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icpc --version
> icpc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
>Reporter: Sam Trahan
>Priority: Major
>
> The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel
> compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in
> gdb reveals that all ten threads exit, and main() is stuck in a wait loop
> calling sleep() forever.
> export CXX=icpc
> export CFLAGS='-fp-model precise'
> export CXXFLAGS='-fp-model precise'
> export CC=icc
> export CPP="icc -E"
> export CXXCPP="icpc -E"
> ./configure --prefix=/some/path
> make
> make check
> Changing the CXXFLAGS to this does not help:
> export CXXFLAGS='-fp-model precise -std=c++11'
> The last bit of output from "make check:"
> make[3]: Entering directory
[jira] [Comment Edited] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[
https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719074#comment-16719074
]
Sam Trahan edited comment on XERCESC-2162 at 12/12/18 3:18 PM:
---
I tried two tests:
1. no optimization - works
2. full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
The fHeartBeat is optimized away, possibly because it was not declared volatile:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
was (Author: samueltrahannoaa):
I tried two tests:
1. no optimization - works
2. full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
where fHeartBeat is never true because it is optimized away:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
> ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
> ---
>
> Key: XERCESC-2162
> URL: https://issues.apache.org/jira/browse/XERCESC-2162
> Project: Xerces-C++
> Issue Type: Bug
> Components: Samples/Tests
>Affects Versions: 3.2.2
> Environment: cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.6 (Maipo)
> uname -a
> Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64
> x86_64 x86_64 GNU/Linux
> Two versions of the Intel compiler suite were tried:
> icpc --version
> icpc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icpc --version
> icpc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
>Reporter: Sam Trahan
>Priority: Major
>
> The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel
> compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in
> gdb reveals that all ten threads exit, and main() is stuck in a wait loop
> calling sleep() forever.
> export CXX=icpc
> export CFLAGS='-fp-model precise'
> export CXXFLAGS='-fp-model precise'
> export CC=icc
> export CPP="icc -E"
> export CXXCPP="icpc -E"
> ./configure --prefix=/some/path
> make
> make check
> Changing the CXXFLAGS to this does not help:
> export CXXFLAGS='-fp-model precise -std=c++11'
> The last bit of output from "make check:"
> make[3]: Entering directory `/a-very-long-path/.../tests'
> PASS: scripts/DOMTest
> PASS: scripts/DOMMemTest
> PASS: scripts/RangeTest
> PASS: scripts/DOMTraversalTest
> XFAIL: scripts/XSerializerTest
> PASS: scripts/XSerializerTest1
> PASS: scripts/XSerializerTest2
> PASS: scripts/XSerializerTest3
> PASS: scripts/XSerializerTest4
> PASS: scripts/XSerializerTest5
> PASS: scripts/XSValueTest
> XFAIL: scripts/InitTermTest
> PASS: scripts/InitTermTest1
> PASS: scripts/InitTermTest2
> PASS: scripts/InitTermTest3
> XFAIL: scripts/ThreadTest
> The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is
> running:
> /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet
> -threads 10 -time 20 personal.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org
[jira] [Commented] (XERCESC-2162) ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
[
https://issues.apache.org/jira/browse/XERCESC-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719074#comment-16719074
]
Sam Trahan commented on XERCESC-2162:
-
I tried two tests:
1. no optimization - works
2. full debugging plus -O3 - fails just the same
(gdb) where
#0 0x7ffe8e57fe2d in nanosleep () from /lib64/libc.so.6
#1 0x7ffe8e57fcc4 in sleep () from /lib64/libc.so.6
#2 0x00409912 in main (argc=-21216, argv=0x7fffad20)
at /a-very-long-path/.../ThreadTest.cpp:1334
The failure is here:
for (threadNum=0; threadNum < gRunInfo.numThreads; threadNum++)
{
if (gThreadInfo[threadNum].fHeartBeat == false) {
where fHeartBeat is never true because it is optimized away:
(gdb) print gThreadInfo[threadNum].fInProgress
value has been optimized out
There is no "volatile" protecting the variable, nor any data structure
containing it. Thus, the compiler is free to optimize it any way it wants to.
> ThreadTest freezes with Intel 17.0.5.239 and 18.0.1.163
> ---
>
> Key: XERCESC-2162
> URL: https://issues.apache.org/jira/browse/XERCESC-2162
> Project: Xerces-C++
> Issue Type: Bug
> Components: Samples/Tests
>Affects Versions: 3.2.2
> Environment: cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.6 (Maipo)
> uname -a
> Linux tfe10 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64
> x86_64 x86_64 GNU/Linux
> Two versions of the Intel compiler suite were tried:
> icpc --version
> icpc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 17.0.5 20170817
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icpc --version
> icpc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
> icc --version
> icc (ICC) 18.0.1 20171018
> Copyright (C) 1985-2017 Intel Corporation. All rights reserved.
>Reporter: Sam Trahan
>Priority: Major
>
> The ThreadTest1 hangs forever when Xerces-C 3.2.2 is compiled using the Intel
> compiler versions 17.0.5.239 or 18.0.1.163. Running ThreadTest1 directly in
> gdb reveals that all ten threads exit, and main() is stuck in a wait loop
> calling sleep() forever.
> export CXX=icpc
> export CFLAGS='-fp-model precise'
> export CXXFLAGS='-fp-model precise'
> export CC=icc
> export CPP="icc -E"
> export CXXCPP="icpc -E"
> ./configure --prefix=/some/path
> make
> make check
> Changing the CXXFLAGS to this does not help:
> export CXXFLAGS='-fp-model precise -std=c++11'
> The last bit of output from "make check:"
> make[3]: Entering directory `/a-very-long-path/.../tests'
> PASS: scripts/DOMTest
> PASS: scripts/DOMMemTest
> PASS: scripts/RangeTest
> PASS: scripts/DOMTraversalTest
> XFAIL: scripts/XSerializerTest
> PASS: scripts/XSerializerTest1
> PASS: scripts/XSerializerTest2
> PASS: scripts/XSerializerTest3
> PASS: scripts/XSerializerTest4
> PASS: scripts/XSerializerTest5
> PASS: scripts/XSValueTest
> XFAIL: scripts/InitTermTest
> PASS: scripts/InitTermTest1
> PASS: scripts/InitTermTest2
> PASS: scripts/InitTermTest3
> XFAIL: scripts/ThreadTest
> The test hangs at that XFAIL: line. The "ps" command reveals ThreadTest1 is
> running:
> /a-very-long-path/.../tests/.libs/lt-ThreadTest -parser=sax -v=never -quiet
> -threads 10 -time 20 personal.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org
