[qpid-proton] branch master updated: PROTON-2293: Stop raw connection wakes from crashing the application - Ensure that the raw connectino is not freed is there is a pending wake. - Ensure that a wake
This is an automated email from the ASF dual-hosted git repository. astitcher pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/qpid-proton.git The following commit(s) were added to refs/heads/master by this push: new ce8a409 PROTON-2293: Stop raw connection wakes from crashing the application - Ensure that the raw connectino is not freed is there is a pending wake. - Ensure that a wake will not be delivered after the application closes the raw connection. - Ensure that wake events will be delivered before disconnected events if both are scheduled to happen. ce8a409 is described below commit ce8a40912d07560ee4008c97988b19fdde0c4d6f Author: Andrew Stitcher AuthorDate: Thu Oct 15 02:04:09 2020 -0400 PROTON-2293: Stop raw connection wakes from crashing the application - Ensure that the raw connectino is not freed is there is a pending wake. - Ensure that a wake will not be delivered after the application closes the raw connection. - Ensure that wake events will be delivered before disconnected events if both are scheduled to happen. --- c/src/proactor/epoll_raw_connection.c| 44 +++- c/src/proactor/raw_connection-internal.h | 1 + c/src/proactor/raw_connection.c | 6 ++--- c/tests/raw_connection_test.cpp | 2 +- 4 files changed, 37 insertions(+), 16 deletions(-) diff --git a/c/src/proactor/epoll_raw_connection.c b/c/src/proactor/epoll_raw_connection.c index ab14b6a..abbd15e 100644 --- a/c/src/proactor/epoll_raw_connection.c +++ b/c/src/proactor/epoll_raw_connection.c @@ -268,19 +268,35 @@ const pn_netaddr_t *pn_raw_connection_remote_addr(pn_raw_connection_t *rc) { void pn_raw_connection_wake(pn_raw_connection_t *rc) { bool notify = false; praw_connection_t *prc = containerof(rc, praw_connection_t, raw_connection); - if (prc) { -lock(&prc->context.mutex); -if (!prc->context.closing) { - prc->waking = true; - notify = wake(&prc->context); -} -unlock(&prc->context.mutex); + lock(&prc->context.mutex); + if (!prc->context.closing) { +prc->waking = true; +notify = wake(&prc->context); } + unlock(&prc->context.mutex); if (notify) wake_notify(&prc->context); } +void pn_raw_connection_close(pn_raw_connection_t *rc) { + praw_connection_t *prc = containerof(rc, praw_connection_t, raw_connection); + lock(&prc->context.mutex); + prc->context.closing = true; + unlock(&prc->context.mutex); + pni_raw_close(rc); +} + static pn_event_t *pni_raw_batch_next(pn_event_batch_t *batch) { - pn_raw_connection_t *raw = &containerof(batch, praw_connection_t, batch)->raw_connection; + praw_connection_t *rc = containerof(batch, praw_connection_t, batch); + pn_raw_connection_t *raw = &rc->raw_connection; + + // Check wake status every event processed + bool waking = false; + lock(&rc->context.mutex); + waking = rc->waking; + rc->waking = false; + unlock(&rc->context.mutex); + if (waking) pni_raw_wake(raw); + return pni_raw_event_next(raw); } @@ -330,9 +346,11 @@ pn_event_batch_t *pni_raw_connection_process(pcontext_t *c, bool sched_wake) { bool wake = false; lock(&c->mutex); c->working = true; - if (sched_wake) wake_done(c); - wake = sched_wake || rc->waking; - rc->waking = false; + if (sched_wake) { +wake_done(c); +wake = rc->waking; +rc->waking = false; + } unlock(&c->mutex); if (wake) pni_raw_wake(&rc->raw_connection); @@ -343,11 +361,13 @@ pn_event_batch_t *pni_raw_connection_process(pcontext_t *c, bool sched_wake) { void pni_raw_connection_done(praw_connection_t *rc) { bool self_notify = false; + bool wake_pending = false; lock(&rc->context.mutex); pn_proactor_t *p = rc->context.proactor; tslot_t *ts = rc->context.runner; rc->context.working = false; self_notify = rc->waking && wake(&rc->context); + wake_pending = rc->waking; unlock(&rc->context.mutex); if (self_notify) wake_notify(&rc->context); @@ -359,7 +379,7 @@ void pni_raw_connection_done(praw_connection_t *rc) { rc->psocket.epoll_io.wanted = wanted; rearm_polling(&rc->psocket.epoll_io, p->epollfd); // TODO: check for error } else { -bool finished_disconnect = raw->rclosed && raw->wclosed && !raw->disconnectpending; +bool finished_disconnect = raw->rclosed && raw->wclosed && !wake_pending && !raw->disconnectpending; if (finished_disconnect) { // If we're closed and we've sent the disconnect then close pni_raw_finalize(raw); diff --git a/c/src/proactor/raw_connection-internal.h b/c/src/proactor/raw_connection-internal.h index 113568c..2e183a0 100644 --- a/c/src/proactor/raw_connection-internal.h +++ b/c/src/proactor/raw_connection-internal.h @@ -94,6 +94,7 @@ bool pni_raw_validate(pn_raw_connection_t *conn); void pni_raw_connected(pn_raw_connection_t *conn); void pni_raw_connect_failed(pn_raw_connection_t *conn); void pni_raw_wake(pn_raw_connection_t *conn)
[qpid-proton] branch master updated: PROTON-2292: TSAN failure in epoll_timer.c. Replace missing lock spotted by TSAN and Coverity.
This is an automated email from the ASF dual-hosted git repository. cliffjansen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/qpid-proton.git The following commit(s) were added to refs/heads/master by this push: new 06833ac PROTON-2292: TSAN failure in epoll_timer.c. Replace missing lock spotted by TSAN and Coverity. 06833ac is described below commit 06833acb10f30343d1e2e970a31e11cb290e32ff Author: Cliff Jansen AuthorDate: Mon Nov 9 00:25:31 2020 -0800 PROTON-2292: TSAN failure in epoll_timer.c. Replace missing lock spotted by TSAN and Coverity. --- c/src/proactor/epoll_timer.c | 1 + 1 file changed, 1 insertion(+) diff --git a/c/src/proactor/epoll_timer.c b/c/src/proactor/epoll_timer.c index 58f0211..6c7c3db 100644 --- a/c/src/proactor/epoll_timer.c +++ b/c/src/proactor/epoll_timer.c @@ -287,6 +287,7 @@ void pni_timer_set(pni_timer_t *timer, uint64_t deadline) { pn_event_batch_t *pni_timer_manager_process(pni_timer_manager_t *tm, bool timeout, bool wake) { uint64_t now = pn_proactor_now_64(); + lock(&tm->context.mutex); tm->context.working = true; if (timeout) tm->timerfd_deadline = 0; - To unsubscribe, e-mail: commits-unsubscr...@qpid.apache.org For additional commands, e-mail: commits-h...@qpid.apache.org
[qpid-dispatch] branch dev-protocol-adaptors-2 updated: DISPATCH-1807: Improve TCP Adaptor self test
This is an automated email from the ASF dual-hosted git repository. chug pushed a commit to branch dev-protocol-adaptors-2 in repository https://gitbox.apache.org/repos/asf/qpid-dispatch.git The following commit(s) were added to refs/heads/dev-protocol-adaptors-2 by this push: new 3a80d73 DISPATCH-1807: Improve TCP Adaptor self test 3a80d73 is described below commit 3a80d73d8866d24f4b1790c99295d3f20793afba Author: Chuck Rolke AuthorDate: Mon Nov 9 18:30:13 2020 -0500 DISPATCH-1807: Improve TCP Adaptor self test * Add a logger that writes component logs to separate log files. * Emit a script that scrapes the logs and produces html result. The current test setup sends one one-byte message from the listening router to the router that is hosting the echo server. This is adequate until all router listeners can forward to any echo server. Later the test may add larger message sizes and concurrent activity. --- tests/system_tests_tcp_adaptor.py | 112 ++ 1 file changed, 90 insertions(+), 22 deletions(-) diff --git a/tests/system_tests_tcp_adaptor.py b/tests/system_tests_tcp_adaptor.py index d8019ef..7d5cc6a 100644 --- a/tests/system_tests_tcp_adaptor.py +++ b/tests/system_tests_tcp_adaptor.py @@ -23,13 +23,14 @@ from __future__ import absolute_import from __future__ import print_function import os +import sys import time import traceback from threading import Event from threading import Timer from system_test import TestCase, Qdrouterd, main_module, TIMEOUT -from system_test import Logger +from system_test import Timestamp from system_test import QdManager from system_test import unittest from system_test import DIR @@ -37,6 +38,13 @@ from system_test import SkipIfNeeded from qpid_dispatch.management.client import Node from subprocess import PIPE, STDOUT +# Tests in this file are organized by classes that inherit TestCase. +# The first instance is TcpAdaptor(TestCase). +# The tests emit files that are named starting with 'TcpAdaptor'. This includes +# logs and shell scripts. +# Subsequent TestCase subclasses must follow this pattern and emit files named +# with the test class name at the beginning of the emitted files. + try: from TCP_echo_client import TcpEchoClient from TCP_echo_server import TcpEchoServer @@ -55,6 +63,47 @@ except ImportError: DISABLE_SELECTOR_TESTS = True DISABLE_SELECTOR_REASON = "Python selectors module is not available on this platform." +class Logger(): +""" +Record event logs as existing Logger. Also add: +* ofile - optional file opened in 'append' mode to which each log line is written +TODO: Replace system_test Logger with this after merging dev-protocol-adaptors branch +""" +def __init__(self, + title="Logger", + print_to_console=False, + save_for_dump=True, + ofilename=None): +self.title = title +self.print_to_console = print_to_console +self.save_for_dump = save_for_dump +self.logs = [] +self.ofilename = ofilename + +def log(self, msg): +ts = Timestamp() +if self.save_for_dump: +self.logs.append( (ts, msg) ) +if self.print_to_console: +print("%s %s" % (ts, msg)) +sys.stdout.flush() +if self.ofilename is not None: +with open(self.ofilename, 'a') as f_out: +f_out.write("%s %s\n" % (ts, msg)) +f_out.flush() + +def dump(self): +print(self) +sys.stdout.flush() + +def __str__(self): +lines = [] +lines.append(self.title) +for ts, msg in self.logs: +lines.append("%s %s" % (ts, msg)) +res = str('\n'.join(lines)) +return res + class TcpAdaptor(TestCase): """ @@ -185,17 +234,10 @@ class TcpAdaptor(TestCase): cls.logger = Logger(title="TcpAdaptor-testClass", print_to_console=True, -save_for_dump=False) +save_for_dump=False, +ofilename='../setUpClass/TcpAdaptor.log') # Write a dummy log line for scraper. -# With this the test log can be identified and consumed in scraper. -# 1. Capture test log output to file 'test.log'. -# 2. Edit away prefix (e.g. '71: ') so each line starts with time of day. -# 3. Edit away ctest lines and fragments that have no time of day. -# 4. Run scraper: -# 'scraper -lm TCP_ADAPTOR,TCP_TEST,ECHO_SERVER,ECHO_CLIENT -f I*.log E*.log test.log > test.html' -# 5. Profit: -# 'firefox test.html' cls.logger.log("SERVER (info) Container Name: TCP_TEST") # Create a scoreboard for the ports @@ -223,6 +265,30 @@ class TcpAdaptor(TestCase): for line in p_out: o_file.wri
[qpid-proton] branch master updated: PROTON-2293: Add missing dummy pn_raw_connection_close functions - Dummy functions for libuv and iocp proactor implementations
This is an automated email from the ASF dual-hosted git repository. astitcher pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/qpid-proton.git The following commit(s) were added to refs/heads/master by this push: new 5a2239e PROTON-2293: Add missing dummy pn_raw_connection_close functions - Dummy functions for libuv and iocp proactor implementations 5a2239e is described below commit 5a2239e4ab552b6067b18b376ab5e6d23fd2470a Author: Andrew Stitcher AuthorDate: Mon Nov 9 20:31:00 2020 -0500 PROTON-2293: Add missing dummy pn_raw_connection_close functions - Dummy functions for libuv and iocp proactor implementations --- c/src/proactor/libuv.c | 1 + c/src/proactor/win_iocp.cpp | 1 + 2 files changed, 2 insertions(+) diff --git a/c/src/proactor/libuv.c b/c/src/proactor/libuv.c index 409d6c5..fa51cc4 100644 --- a/c/src/proactor/libuv.c +++ b/c/src/proactor/libuv.c @@ -1355,5 +1355,6 @@ pn_raw_connection_t *pn_raw_connection(void) { return NULL; } void pn_proactor_raw_connect(pn_proactor_t *p, pn_raw_connection_t *rc, const char *addr) {} void pn_listener_raw_accept(pn_listener_t *l, pn_raw_connection_t *rc) {} void pn_raw_connection_wake(pn_raw_connection_t *conn) {} +void pn_raw_connection_close(pn_raw_connection_t *conn) {} const struct pn_netaddr_t *pn_raw_connection_local_addr(pn_raw_connection_t *connection) { return NULL; } const struct pn_netaddr_t *pn_raw_connection_remote_addr(pn_raw_connection_t *connection) { return NULL; } diff --git a/c/src/proactor/win_iocp.cpp b/c/src/proactor/win_iocp.cpp index 88737e4..a71947a 100644 --- a/c/src/proactor/win_iocp.cpp +++ b/c/src/proactor/win_iocp.cpp @@ -3426,5 +3426,6 @@ pn_raw_connection_t *pn_raw_connection(void) { return NULL; } void pn_proactor_raw_connect(pn_proactor_t *p, pn_raw_connection_t *rc, const char *addr) {} void pn_listener_raw_accept(pn_listener_t *l, pn_raw_connection_t *rc) {} void pn_raw_connection_wake(pn_raw_connection_t *conn) {} +void pn_raw_connection_close(pn_raw_connection_t *conn) {} const struct pn_netaddr_t *pn_raw_connection_local_addr(pn_raw_connection_t *connection) { return NULL; } const struct pn_netaddr_t *pn_raw_connection_remote_addr(pn_raw_connection_t *connection) { return NULL; } - To unsubscribe, e-mail: commits-unsubscr...@qpid.apache.org For additional commands, e-mail: commits-h...@qpid.apache.org
[qpid-proton] branch master updated: PROTON-2292: update tsan.supp for new epoll proactor timer code
This is an automated email from the ASF dual-hosted git repository. cliffjansen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/qpid-proton.git The following commit(s) were added to refs/heads/master by this push: new 8c8a8bc PROTON-2292: update tsan.supp for new epoll proactor timer code 8c8a8bc is described below commit 8c8a8bc14cb5a24382bbb08e42bc1a7b69188a95 Author: Cliff Jansen AuthorDate: Mon Nov 9 23:44:18 2020 -0800 PROTON-2292: update tsan.supp for new epoll proactor timer code --- tests/tsan.supp | 4 1 file changed, 4 insertions(+) diff --git a/tests/tsan.supp b/tests/tsan.supp index f0f9020..5a1c61a 100644 --- a/tests/tsan.supp +++ b/tests/tsan.supp @@ -30,3 +30,7 @@ race:cpp/examples/broker # found by threaderciser, in c/src/proactor/epoll.c race:^listener_final_free$ race:^pn_proactor_connect2$ + +# PROTON-2292 & PROTON-1496: false suspected collision between set/free +mutex:^pni_timer_set$ +mutex:^pni_timer_free$ - To unsubscribe, e-mail: commits-unsubscr...@qpid.apache.org For additional commands, e-mail: commits-h...@qpid.apache.org