[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295449#comment-17295449
]
Clifford Jansen commented on PROTON-2231:
-
The previous speculative remark fails after an instrumented run and inspection
of the libuv code. back to the drawing board.
In other news, this eisen-bug goes into deep hiding if rr is added to the debug
mix.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257900#comment-17257900
]
Clifford Jansen commented on PROTON-2231:
-
I took a look at the stack of a run that hung in pn_proactor_wait/uv_run.
I see the p->disconnect boolean is set and the proactor is stuck unable to
process the pending disconnect operation. My guess is that
static inline void notify(pn_proactor_t* p) {
uv_async_send(>notify);
}
does not wake the "leader" thread in the following circumstance:
L thread is prepping to do uv_run()
X thread uv_async_send part 1: register uv callback (no-op since NULL
callback provided)
X thread uv_async_sync part 2: wake uv loop (also no-op, not currently
running)
L thread uv_run()
Whereas if the uv_run() wins the race against uv_async_send(), the run loop
returns and the disconnect cleanup activity winds things up as expected.
All speculation, but if relevant, there probably should be a proactor callback
that temporarily sets the next uv_run_mode to UV_RUN_NOWAIT (with appropriate
synchronization).
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254516#comment-17254516
]
Roddie Kieley commented on PROTON-2231:
---
I've now tested in the same way on both an 8 core atom Fedora 32 box as well as
the 6 core macOS box. The behaviour with libuv is broken on both while the
epoll proactor implementation on the Fedora 32 box is well behaved every run.
Looking more closely at libuv we see that the problem behaviours, i.e. stopping
new listening, hanging on exit, or asserting as per the description, appear to
be caused by the fact that at some point in the test execution uv_run does not
return when called in the chain pn_proactor_wait -> leader_lead_lh -> uv_run.
This causes the current leader to not relinquish the lead and the other threads
to be unable to do more work or exit.
Temporarily switching to pn_proactor_get instead of pn_proactor_wait for the
test shows that execution runs successfully until the end when leader_lead_lh
again blocks at uv_run, does not exit, cannot join, and thus the test is still
hung although main test execution behaviour appears correct. If we add a
uv_loop_alive check before the uv_run this check passes, i.e. returns != 0, and
uv_run continues to cause the problem.
As the libuv documentation says that for
[UV_RUN_ONCE|http://docs.libuv.org/en/v1.x/loop.html]
{quote}Note that this function blocks if there are no pending callbacks.
{quote}
Further investigation will be along those lines, however suggestions welcome
[~astitcher] or [~cliffjansen].
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252077#comment-17252077
]
ASF subversion and git services commented on PROTON-2231:
-
Commit e4c179855fef161e656ba5ab422c1c24a2df9ded in qpid-proton's branch
refs/heads/master from Roddie Kieley
[ https://gitbox.apache.org/repos/asf?p=qpid-proton.git;h=e4c1798 ]
NO-JIRA: github actions CI for macOS disabled threaderciser test due to
PROTON-2231.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241652#comment-17241652
]
Andrew Stitcher commented on PROTON-2231:
-
This piece of code is keeping track of the total number of proactor
connections/listeners etc. open so that the proactor can generate an idle event
when there are none left.
So I think it is likely that it should probably not be executing for a close of
a connection that is already closed.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241412#comment-17241412
]
Roddie Kieley commented on PROTON-2231:
---
No, I don't think so. It sounds like what we are saying here is the fact that
the code executes an assert like:
{code:java}
static void remove_active_lh(pn_proactor_t *p) {
assert(p->active > 0);
if (--p->active == 0) {
p->need_inactive = true;
}
} {code}
when you attempt to close and remove more listeners than have been opened is
incorrect based on the above. Either that code shouldn't execute at that point
in time or the assert itself is invalid.
I will review the other implementations and the documentation and see about
next steps to address the issue.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240977#comment-17240977
]
Andrew Stitcher commented on PROTON-2231:
-
[~rkieley] Thank you for the debugging. However I think this is a misdiagnosis
of the issue:
The documentation for pn_listener_close() has nothing to say about how many
time it may be called so it may be called as many time as you like, except that
it will only close the one listener! However you are not allowed to call
pn_listener_close() after you have received the PN_LISTEN_CLOSE event for the
listener.
I think it is a totally legitimate test to call pn_listener_close() multiple
times for the listener. And the epoll implementation seems to fare correctly
under this test. So I'd say this has pointed out an actual bug in the libuv
proactor, not a bug in threadercizer.
Did I miss something or misunderstand?
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240177#comment-17240177
]
Roddie Kieley commented on PROTON-2231:
---
Assuming that the assert in remove_active_lh is correct and that active
requires being greater than zero then then the problem appears to be that the
threaderciser does not attempt to adhere to that rule. By that I mean that it
will attempt to close more listeners than were opened thus ensuring that the
number of active listeners becomes less than zero.
At the moment I have a partial fix with which it appears in that over the
course of 20 runs we never see it assert during the run but at the moment in
shutting down it can either hang or infrequently assert, which was also
observed in the unmodified code. See my [PROTON-2231
|https://github.com/apache/qpid-proton/compare/master...RoddieKieley:PROTON-2231]
branch, commit
[49efdd5c17d6f5f0b83ec9820da54841df18145d|https://github.com/RoddieKieley/qpid-proton/commit/49efdd5c17d6f5f0b83ec9820da54841df18145d]
in particular. Needs a bit more work but appears to be on the right track.
Here's snippet from a failed run:
{code}
.
.
.
(7556e000) [0x7f971b80de00] close-listen
(755f1000) [0x7f971b80de00] PN_LISTEN_CLOSE on :::53389, opened: 0
(7505) [0x7f9719200100] wake
(7525c000) [0x7f9718500330] connect
(75468000) [0x7f9718600c20] wake
(75156000) [0x7f971b80d800] listen
(75156000) [0x7f9719100850] wake
(755f1000) [0x7f971b80d800] PN_LISTEN_OPEN on :::53396, opened: 1
(75362000) [0x7f971b801600] listen
(750d3000) [0x7f971b801600] PN_LISTEN_OPEN on :::53397, opened: 2
(75468000) [0x7f971b80ea00] listen
(753e5000) [0x7f971b80ea00] PN_LISTEN_OPEN on :::53398, opened: 3
(75362000) [0x7f97184036c0] wake
(7556e000) [0x7f9718600570] wake
(7556e000) [0x7f9718600c20] wake
(75468000) [0x7f971b801600] close-listen
(752df000) [0x7f971b801600] PN_LISTEN_CLOSE on :::53397, opened: 2
(75156000) [0x7f9719200100] wake
(7505) [0x7f9719100850] wake
(7525c000) [0x7f9718809800] listen
(7525c000) [0x7f9718500330] wake
(754eb000) [0x7f9718809800] PN_LISTEN_OPEN on :::53399, opened: 3
(7525c000) [0x7f9719200100] wake
(7556e000) [0x7f971c007e00] close-listen
(754eb000) [0x7f971c007e00] PN_LISTEN_CLOSE on :::53371, opened: 2
(75362000) [0x7f9718404a70] connect
(75362000) [0x7f97184036c0] wake
(75468000) [0x7f9718602830] connect
(75468000) [0x7f971b805600] listen
(750d3000) [0x7f971b805600] PN_LISTEN_OPEN on :::53402, opened: 3
(750d3000) [0x7f9718600e30] connect
(10d6625c0) shut down
(75156000) user_thread end
(7525c000) user_thread end
(7505) user_thread end
(75362000) user_thread end
(7556e000) user_thread end
(75468000) user_thread end
(10d6625c0) disconnect
(755f1000) [0x7f971b805600] PN_LISTEN_CLOSE on :::53402, opened: 2
(751d9000) [0x7f971b80ea00] PN_LISTEN_CLOSE on :::53398, opened: 1
(755f1000) [0x7f971c000600] PN_LISTEN_CLOSE on :::53392, opened: 0
(753e5000) [0x7f971b80d800] PN_LISTEN_CLOSE on :::53396, opened: -3
(750d3000) [0x7f9718809800] PN_LISTEN_CLOSE on :::53399, opened: -1
(752df000) [0x7f971b008c00] PN_LISTEN_CLOSE on :::53391, opened: -2
Assertion failed: (p->active > 0), function remove_active_lh, file
/wip/src/github.com/apache/qpid-proton/c/src/proactor/libuv.c, line 394.
{code}
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17239854#comment-17239854
]
Roddie Kieley commented on PROTON-2231:
---
I think everything I've seen indicates that it is not platform specific.
Tracing through the function calls with the timeouts disabled we see that it
always the code path always comes directly through remove_active which
eventually asserts that active must be greater than zero for you to call a
remove that will decrement it. As far as I can tell the threadercizer does not
track open versus closed listeners and can close more than have been opened,
although at this point this is from recollection.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235535#comment-17235535
]
Andrew Stitcher commented on PROTON-2231:
-
I've now also seen this same issue on Linux too. So I'm fairly sure it is
actually an issue in the libuv proactor generally and not just a platform
issue. Of course there may be something about macOS/FreeBSD that make it happen
more frequently.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: freebsd, macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234950#comment-17234950
]
Andrew Stitcher commented on PROTON-2231:
-
I've also seen this fairly commonly running this test on FreeBSD 12 as well -
This makes it look to me like it's an issue with the libuv proactor rather than
the macOS platform.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227570#comment-17227570
]
Roddie Kieley commented on PROTON-2231:
---
After collecting lots of logging using the -debug method via:
{code}
./c-threaderciser -time 10 -threads 12 -debug -no-timeout -no-cancel-timeout
{code}
A very short reproduction of the assertion was recorded:
{code}
i7mbp:tests rkieley$ ./c-threaderciser -time 10 -threads 12 -debug -no-timeout
-no-cancel-timeout 2>&1 >>
/wip/issues/issues.apache.org/PROTON-2231/threader-110620-4-no-timeouts-1.out |
tee
/wip/issues/issues.apache.org/PROTON-2231/threader-110620-4-no-timeouts-1.out
(1053a35c0) [0x7fb3c8802800] listen
(709b) user_thread start
(70a33000) proactor_thread start
(70ab6000) user_thread start
(70b39000) proactor_thread start
(70c3f000) proactor_thread start
(709b) [0x7fb3c7d006a0] connect
(70cc2000) user_thread start
(70bbc000) user_thread start
(70d45000) proactor_thread start
(70dc8000) user_thread start
(70bbc000) [0x7fb3c9100070] connect
(70e4b000) proactor_thread start
(70ece000) user_thread start
(70f51000) proactor_thread start
(709b) [0x7fb3c8802800] close-listen
(709b) [0x7fb3c9100070] wake
(70dc8000) [0x7fb3c7d01f10] connect
(70dc8000) [0x7fb3c980d200] listen
(70dc8000) [0x7fb3c8802800] close-listen
(70dc8000) [0x7fb3c9100070] wake
(70ece000) [0x7fb3c7f02eb0] connect
(709b) [0x7fb3c980de00] listen
(70bbc000) [0x7fb3c7d006a0] wake
(70ece000) [0x7fb3c980d200] close-listen
(70cc2000) [0x7fb3c7d006a0] wake
(70bbc000) [0x7fb3c7f041f0] connect
(70bbc000) [0x7fb3c7f02eb0] wake
(70ece000) [0x7fb3c9100070] wake
(70a33000) [0x7fb3c8802800] listening on :::62859
(70a33000) [0x7fb3c7c04e60] connect
(70c3f000) [0x7fb3c980de00] listening on :::62860
(70c3f000) [0x7fb3c9101200] connect
(70dc8000) [0x7fb3c7f02c90] connect
(70dc8000) [0x7fb3c7f02c90] wake
(709b) [0x7fb3c7f02c90] wake
(70bbc000) [0x7fb3c980de00] close-listen
Assertion failed: (p->active > 0), function remove_active_lh, file
/wip/src/github.com/apache/qpid-proton/c/src/proactor/libuv.c, line 394.
i7mbp:tests rkieley$
{code}
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227457#comment-17227457
]
Roddie Kieley commented on PROTON-2231:
---
Investigation ongoing. Seeing two variations of the hang at the end of the
threaderciser using the following on a 6 core i7 based mac:
{code}
for i in {1..10}; do `echo echo $i; date +%H:%M:%S; ./c-threaderciser -time 10
-threads 12`; done;
{code}
This hangs the very first time it runs in many cases in one of two places
{code}
for (i = 0; i < threads/2; ++i) pthread_join(proactor_threads[i], );
{code}
with 6 of the 7 threads sitting in pn_proactor_wait
OR threaderciser hangs at
{code}
global_destroy()
{code}
which itself is hung on
{code}
pn_proactor_free(g->proactor);
{code}
As well I also see the Assertion failed from the above comment periodically.
More investigation and understanding required to put the pieces together.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215708#comment-17215708
]
Roddie Kieley commented on PROTON-2231:
---
While I have not yet attempted to reproduce using libuv on other platforms,
either Linux or Windows, I can now confirm local reproduction on macOS 10.14
using today's latest code:
{code}
i7mbp:1 rkieley$ ctest -VV -R c-threaderciser --timeout 60
UpdateCTestConfiguration from
:/wip/builds/github.com/apache/qpid-proton/101620/1/DartConfiguration.tcl
Parse Config
file:/wip/builds/github.com/apache/qpid-proton/101620/1/DartConfiguration.tcl
UpdateCTestConfiguration from
:/wip/builds/github.com/apache/qpid-proton/101620/1/DartConfiguration.tcl
Parse Config
file:/wip/builds/github.com/apache/qpid-proton/101620/1/DartConfiguration.tcl
Test project /wip/builds/github.com/apache/qpid-proton/101620/1
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 6
Start 6: c-threaderciser
6: Test command: /opt/local/bin/python
"/wip/src/github.com/apache/qpid-proton/scripts/env.py" "--"
"/wip/builds/github.com/apache/qpid-proton/101620/1/c/tests/c-threaderciser"
6: Test timeout computed to be: 60
6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
connect, close-connect, wake, timeout, cancel-timeout]
6: Assertion failed: (p->active > 0), function remove_active_lh, file
/wip/src/github.com/apache/qpid-proton/c/src/proactor/libuv.c, line 394.
1/1 Test #6: c-threaderciser ..***Failed1.16 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 1.17 sec
The following tests FAILED:
6 - c-threaderciser (Failed)
Errors while running CTest
i7mbp:1 rkieley$
{code}
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: proton-c-0.32.0
>Reporter: Jiri Daněk
>Assignee: Roddie Kieley
>Priority: Major
> Labels: macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-2231) Assertion fail on macOS with libuv in c-threaderciser test
[
https://issues.apache.org/jira/browse/PROTON-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182316#comment-17182316
]
Jiri Daněk commented on PROTON-2231:
I've split out the issue with the test getting stuck, because it appears to be
not limited solely to libuv/macOS. New issue is PROTON-2273.
> Assertion fail on macOS with libuv in c-threaderciser test
> --
>
> Key: PROTON-2231
> URL: https://issues.apache.org/jira/browse/PROTON-2231
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Reporter: Jiri Daněk
>Priority: Major
> Labels: macOS
>
> As described on PROTON-2225, the test fails with assertion error. It is
> currently disabled on macOS for this reason (in .travis.yml).
> {noformat}
> 6: Test command: /usr/local/opt/python/libexec/bin/python
> "/Users/travis/build/jiridanek/qpid-proton/scripts/env.py" "--"
> "/Users/travis/build/jiridanek/qpid-proton/build/c/tests/c-threaderciser"
> 6: Test timeout computed to be: 1500
> 6: threaderciser start: threads=8, time=1, actions=[listen, close-listen,
> connect, close-connect, wake, timeout, cancel-timeout]
> 6: Assertion failed: (p->active > 0), function remove_active_lh, file
> /Users/travis/build/jiridanek/qpid-proton/c/src/proactor/libuv.c, line 392.
> 6/31 Test #6: c-threaderciser ..***Failed0.18 sec
> {noformat}
> If the test is meant to stay disabled for a longer time, it will have to be
> disabled in CMakeLists.txt, so that users compiling the project do not run it
> accidentally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
