On Mon, Oct 24, 2022 at 10:39:45AM +0200, Jean Baptiste Favre wrote:
> Please provide some more details on you issue:
> * which plugin do you use? Is there any specific configuration?
I am not YunQiang Su but I'm pretty sure we're talking about the same
thing. Any remap plugin will fail to load; for example, add the
following to /etc/trafficserver/remap.config (example from the manual):
map http://a.com http://b.com @plugin=regex_remap.so @pparam=maps.reg
With contents of /etc/trafficserver/maps.reg (again from the manual):
^/(ogre.*)/more http://www.ogre.com/$h/$0/$1
Global plugins are not affected by this issue.
> * do you have some trafficserver's logs to provide?
Here you are (/var/log/trafficserver/diags.log):
[Dec 28 20:00:59.369] traffic_server ERROR: plugin 'regex_remap.so' failed to
load:
/run/trafficserver/adb67a84-e739-4e8f-bc9d-c4797cec16e4/usr/lib/trafficserver/modules/regex_remap.so:
failed to map segment from shared object
[Dec 28 20:00:59.369] traffic_server ERROR: [ReverseProxy] failed to add remap
rule at /etc/trafficserver/remap.config line 199:
/run/trafficserver/adb67a84-e739-4e8f-bc9d-c4797cec16e4/usr/lib/trafficserver/modules/regex_remap.so:
failed to map segment from shared objectfailed to remove runtime copy: Success
[Dec 28 20:00:59.369] traffic_server WARNING: something failed during
BuildTable() -- check your remap plugins!
[Dec 28 20:00:59.369] traffic_server FATAL: remap.config failed to load
[Dec 28 19:46:23.246] traffic_server ERROR: plugin 'cachekey.so' failed to
load:
/run/trafficserver/c4edf320-f56b-4193-8554-7db2dccfa0a7/usr/lib/trafficserver/modules/cachekey.so:
failed to map segment from shared object
[Dec 28 19:46:23.246] traffic_server ERROR: [ReverseProxy] failed to add remap
rule at /etc/trafficserver/remap.config line 197:
/run/trafficserver/c4edf320-f56b-4193-8554-7db2dccfa0a7/usr/lib/trafficserver/modules/cachekey.so:
failed to map segment from shared objectfailed to remove runtime copy: Success
[Dec 28 19:46:23.246] traffic_server WARNING: something failed during
BuildTable() -- check your remap plugins!
[Dec 28 19:46:23.246] traffic_server FATAL: remap.config failed to load
There are errors from dlopen. As the OP said, the culprit is that it
copies the plugins from /usr/lib to /run and tries to dlopen them
there. This is doomed to fail because, exactly as the OP said, /run
is mounted with the "noexec" option. Upstream logic behind this is to
dynamically reload plugins that are changed while the server is
running. I guess it makes sense with custom plugins and some other
scenarios but there's no way for this to work on Debian systems.
The attached dirty hack removes this code but also leads to test
failures of two test programs where this dynamic reloading
functionality is tested. It is not a proper fix but at least makes
remap plugins load again.
Like the OP, I also believe this bug deserves RC severity.
Additionally (and this is an entirely separate issue, completely
unrelated with the problem described above), some global plugins fail
to load as well, for various reasons:
[Dec 28 19:34:21.066] traffic_server NOTE: loading plugin '/usr/lib/trafficserve
r/modules/prefetch.so'
[Dec 28 19:34:21.066] traffic_server ERROR: unable to find TSPluginInit
function in '/usr/lib/trafficserver/modules/prefetch.so':
/usr/lib/trafficserver/modules/prefetch.so: undefined symbol: TSPluginInit
[Dec 28 19:34:21.066] traffic_server FATAL: unable to find TSPluginInit
function in '/usr/lib/trafficserver/modules/prefetch.so':
/usr/lib/trafficserver/modules/prefetch.so: undefined symbol: TSPluginInit
There is indeed no such function, and according to the plugin API
there should be.
[Dec 29 14:31:12.743] traffic_server ERROR: [geoip_acl] No Geo library
available!
[Dec 29 14:31:12.743] traffic_server ERROR: [ReverseProxy] failed to add remap
rule at /etc/trafficserver/remap.config line 206: failed to initialize plugin
geoip_acl.so: Unknown plugin errorfailed to remove runtime copy: Success
[Dec 29 14:31:12.744] traffic_server WARNING: something failed during
BuildTable() -- check your remap plugins!
[Dec 29 14:31:12.744] traffic_server FATAL: remap.config failed to load
I have tested only those that are pcre-based.
--- trafficserver-9.2.3+ds.orig/proxy/http/remap/PluginDso.cc
+++ trafficserver-9.2.3+ds/proxy/http/remap/PluginDso.cc
@@ -72,7 +72,7 @@
result = false;
} else {
PluginDebug(_tag, "plugin '%s' effective path: %s", _configPath.c_str(), _effectivePath.c_str());
-
+#if 0
/* Copy the installed plugin DSO to a runtime directory if dynamic reload enabled */
std::error_code ec;
if (isDynamicReloadEnabled() && !copy(_effectivePath, _runtimePath, ec)) {
@@ -88,9 +88,9 @@
fs::file_status fs = fs::status(_effectivePath, ec);
_mtime = fs::modification_time(fs);
PluginDebug(_tag, "plugin '%s' modification time %ld", _configPath.c_str(), _mtime);
-
+#endif
/* Now attempt to load the plugin DSO */
- if ((_dlh = dlopen(_runtimePath.c_str(), RTLD_NOW | RTLD_LOCAL)) == nullptr) {
+ if ((_dlh = dlopen(_effectivePath.c_str(), RTLD_NOW | RTLD_LOCAL)) == nullptr) {
#if defined(freebsd) || defined(openbsd)
char *err = (char *)dlerror();
#else
@@ -105,12 +105,13 @@
PluginError("plugin '%s' failed to load: %s", _configPath.c_str(), error.c_str());
}
}
-
+#if 0
/* Remove the runtime DSO copy even if we succeed loading to avoid leftovers after crashes */
if (_preventiveCleaning) {
clean(error);
}
- }
+#endif
+
PluginDebug(_tag, "plugin '%s' finished loading DSO", _configPath.c_str());
return result;
--- trafficserver-9.2.3+ds.orig/proxy/http/remap/Makefile.am
+++ trafficserver-9.2.3+ds/proxy/http/remap/Makefile.am
@@ -80,32 +80,7 @@
$(CXX_Clang_Tidy)
TESTS = $(check_PROGRAMS)
-check_PROGRAMS = test_PluginDso test_PluginFactory test_RemapPluginInfo test_NextHopStrategyFactory test_NextHopRoundRobin test_NextHopConsistentHash
-
-test_PluginDso_CPPFLAGS = $(AM_CPPFLAGS) -I$(abs_top_srcdir)/tests/include -DPLUGIN_DSO_TESTS
-test_PluginDso_LIBTOOLFLAGS = --preserve-dup-deps
-EXTRA_test_PluginDso_DEPENDENCIES = unit-tests/plugin_v1.la
-test_PluginDso_LDADD = $(COMMON_PLUGINDSO_LDADDS)
-test_PluginDso_LDFLAGS = $(AM_LDFLAGS)
-test_PluginDso_SOURCES = \
- unit-tests/test_PluginDso.cc \
- unit-tests/plugin_testing_common.cc \
- PluginDso.cc
-
-test_PluginFactory_CPPFLAGS = $(AM_CPPFLAGS) -I$(abs_top_srcdir)/tests/include -DPLUGIN_DSO_TESTS
-test_PluginFactory_LIBTOOLFLAGS = --preserve-dup-deps
-EXTRA_test_PluginFactory_DEPENDENCIES = \
- unit-tests/plugin_v1.la \
- unit-tests/plugin_init_fail.la \
- unit-tests/plugin_instinit_fail.la
-test_PluginFactory_LDADD = $(COMMON_PLUGINDSO_LDADDS)
-test_PluginFactory_LDFLAGS = $(AM_LDFLAGS)
-test_PluginFactory_SOURCES = \
- unit-tests/test_PluginFactory.cc \
- unit-tests/plugin_testing_common.cc \
- PluginFactory.cc \
- PluginDso.cc \
- RemapPluginInfo.cc
+check_PROGRAMS = test_RemapPluginInfo test_NextHopStrategyFactory test_NextHopRoundRobin test_NextHopConsistentHash
test_RemapPluginInfo_CPPFLAGS = $(AM_CPPFLAGS) -I$(abs_top_srcdir)/tests/include -DPLUGIN_DSO_TESTS
test_RemapPluginInfo_LIBTOOLFLAGS = --preserve-dup-deps
