(trafficserver) branch master updated: Fix an intermittent cache unit test deadlock at exit (#13535)

[bneradt]

This is an automated email from the ASF dual-hosted git repository.

bneradt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/master by this push:
     new 0c18d46bc6 Fix an intermittent cache unit test deadlock at exit 
(#13535)
0c18d46bc6 is described below

commit 0c18d46bc6fcc311ac94d2ada472f6611ea4a01d
Author: Brian Neradt <brian.ner...@gmail.com>
AuthorDate: Tue Aug 11 13:14:24 2026 -0500

    Fix an intermittent cache unit test deadlock at exit (#13535)
    
    The cache unit test harness starts the event and net processors but
    never stops them, so every test binary reaches exit() with ET_NET
    threads still running. Static destruction then frees globals out from
    under those threads: the records table in RecCore.cc is destroyed while
    a still-initializing event thread reads it through RecGetRecordInt(),
    and the ts::Metrics storage blob is released while NetHandler's activity
    loop increments a counter into it. Both are heap-use-after-frees, and
    under ASan the reporting thread races the exiting main thread. Usually
    the process dies first and the report is truncated to two lines with a
    zero exit status, so ctest reports a pass; occasionally the report
    deadlocks instead and the test hangs until ctest times it out after
    1500 seconds. The short tests that never touch the cache lose this race
    most often, which is why CacheAggregateWriteBuffer and CacheStripe are
    the ones that fail.
    
    This addresses the deadlock at its source by giving the harness's Catch2
    listener a testRunEnded hook that shuts the event system down and joins
    the event threads before the test binary returns from main. Once the
    threads are gone, static destruction has no concurrent reader to race,
    so neither use-after-free can be reported and the ASan reporting
    deadlock cannot arise.
    
    Co-authored-by: Claude Opus 5 <nore...@anthropic.com>
---
 src/iocore/cache/unit_tests/main.cc | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/src/iocore/cache/unit_tests/main.cc 
b/src/iocore/cache/unit_tests/main.cc
index 6b76c504d2..d07bb32f12 100644
--- a/src/iocore/cache/unit_tests/main.cc
+++ b/src/iocore/cache/unit_tests/main.cc
@@ -162,6 +162,20 @@ struct EventProcessorListener : Catch::EventListenerBase {
     std::string src_dir       = std::string(TS_ABS_TOP_SRCDIR) + 
"/src/iocore/cache/unit_tests/etc/";
     Layout::get()->sysconfdir = std::move(src_dir);
   }
+
+  // Every test binary using this harness reaches exit() with the event threads
+  // still running, so stop them and wait for them before static destruction
+  // frees the globals they read.
+  void
+  testRunEnded(Catch::TestRunStats const & /* stats ATS_UNUSED */) override
+  {
+    TSSystemState::shut_down_event_system();
+    for (EThread *ethread : eventProcessor.active_ethreads()) {
+      if (ethread->tid != ink_thread_null()) {
+        ink_thread_join(ethread->tid);
+      }
+    }
+  }
 };
 CATCH_REGISTER_LISTENER(EventProcessorListener);