"Currently there is no way for a plugin to just reload its config file when it is running as a remap"
Plugin Messages work for me - https://docs.trafficserver.apache.org/en/8.0.x/developer-guide/api/functions/TSLifecycleHookAdd.en.html#_CPPv3N17TSLifecycleHookID21TS_LIFECYCLE_MSG_HOOKE That's what TxnBox uses to signal a config reload. Also, as of ATS 9 and the new sets of remap plugin callbacks that happen at.the "remap.config" loading time, different instances of the same plugin in different rules can share state, and the timed probes could be based there (TxnBox uses this to share config files, so any particular config file is loaded and parsed into YAML only once - stole that from Zwoop). That said, it doesn't handle the case of changing the remap rule arguments, and does require config buffering logic inside the plugin which isn't always easy. In terms of reloading just a config for a plugin, this gets a bit intricate when the plugin is used in multiple remap rules each of which uses a subset of a number of configuration files. Does the core handle each of those, substituting it in *and* dealing with config buffering around live traffic? How would that work - would the core do a RemapNewInstance and RemapDestroyInstance? One reason TxnBox has just "reload" is I didn't want to deal with this complexity and possible cross config dependencies. I'm not enthused about trying to put that in the core and then debug it. If we're going to do anything, I'd go with Evan's plan and allow a plugin to add files that trigger a "remap.config" reload. It's not something that's required if it doesn't work in your deployment, it's simple to understand and implement. If you really need better, there are (as noted above) ways to do that. On Tue, Jul 21, 2020 at 6:33 PM Evan Zelkowitz <[email protected]> wrote: > Ok well in that case maybe this does make more sense to integrate as a > part of the plugindso code via its own APIs, such as the ones to > notify of a full remap reload pre/post. Instead have a new api in > there of a per-plugin config file that it can keep track of and notify > the plugin who registered it for updates > > Though the includes issue will have to be handled elsewhere > > On Tue, Jul 21, 2020 at 5:22 PM Sudheer Vinukonda > <[email protected]> wrote: > > > > I don't think periodic reload of config for regex_revalidate is > optional. Pretty certain we used that functionality in the past (for > purging tombstones that get added on the fly). > > > > Reloading just the config is not the same as reloading the remap DSO or > conflicts with the remap reload even. > > > > The concern with having to reload the entire remap file is that, it is > "unwarranted" and will result in reloading a giant remap file (for e.g our > remap file is 20K+ lines long) for a change in some random plugin's config > on a single remap line. > > > > > > > > > > On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz < > [email protected]> wrote: > > > > > > regex reval first uses the tsmgmtupdate call in all cases, so it will > > be triggered by a config reload and since it's a global plugin it can > > do that, the periodic reload is optional. During the last summit > > there was a lot of concern over remap.config reloading on its own > > outside of config reload due to a bug that existed. Just having timed > > reloads would basically reintroduce that functionality just for that > > plugin for its configuration. Doing it this way also creates one > > single way to load a config, a plugin doesn't have to know if its > > reloading a config or monitoring anything, it is just destroyed and > > recreated and loads everything as it normally would in newinstance > > > > You also don't have to create duplicate reload code, or reinvent it, > > it would be a simple API call to register a file that should be > > monitored and that's it > > > > What is the concern with doing a remap reload in these cases though? > > Since it would essentially have to do the same thing if a plugin did > > not use a config file and had all of its parameters listed out as > > pparams to a plugin that would have changed a line in the remap > > config. So to me a plugins config file is equivalent to packaging up a > > bunch of pparams or having them represented in a different/logical > > way, but are still equivalent to a line of a bunch of pparams > > > > On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda > > <[email protected]> wrote: > > > > > > Hmm..I am not sure to understand the concern with reloading > individual config files outside of "config reload". > > > I think a reasonable approach is to use a background thread > periodically polling for config file changes and load it in isolation (an > example is regex_revalidate plugin). > > > > https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c > > > What are the concerns with that approach? > > > > > > > > > > > > On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz < > [email protected]> wrote: > > > > > > Currently there is no way for a plugin to just reload its config file > > > when it is running as a remap, unless you are running some sort of > > > timer or checking on every remap call. The mgmtupdate cannot be used > > > for remaps due to how it is designed. Using a timer you then start > > > reloading configs outside of the specified config reload call which > > > many do not like (since we had the previous issue of the remap > > > reloading on its own when it was changed). I suggest reloading the > > > remap because that keeps this within a well designed workflow that we > > > already know wrt to reference counting and deletion/instantiation of > > > plugins and what data they will be loaded with > > > > > > I suppose another option would be to add functionality to the > > > PluginDSO and have it do file monitoring as requested by plugins to > > > reload individual ones rather than just the blanket remap reload, but > > > currently anything dealing with config files like that is already > > > handled by the manager, this would be a large extension to the plugin > > > code > > > > > > On Tue, Jul 21, 2020 at 4:24 PM John Rushford <[email protected]> > wrote: > > > > > > > > IMO a config file change should just cause the config to be reloaded > no the entire remap or plugin DSO. > > > > > > > > Thanks > > > > John > > > > > > > > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda < > [email protected]> wrote: > > > >> > > > >> I think it makes sense when "includes" are changed, but, I'm > wondering if it makes sense to reload the remap file when a specific config > file for a plugin is changed. > > > >> > > > >> Shouldn't a plugin config file change limit to reloading the plugin > (or even just the config file for that plugin) as opposed to reloading the > entire remap config? > > > >> > > > >> Thanks, > > > >> > > > >> Sudheer > > > >> > > > >> > > > >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz < > [email protected]> wrote: > > > >> > > > >> > > > >> There is an issue surrounding remap reloading when includes and > plugin > > > >> config files are used. These will not trigger a remap reload on > their > > > >> own. I think at this point everyone uses their external management > > > >> tools to issue a touch to the remap file to trigger a reload to > > > >> happen, but it's a bit of a work-around. > > > >> > > > >> I'm proposing adding an API that will allow plugins to register > their > > > >> config files to be monitored in the same way that remap and records > > > >> are currently monitored. When a traffic_ctl config reload is > issued, > > > >> if one of these files has changed that can also trigger a remap > > > >> reload. > > > >> > > > >> I have a POC going currently where I can add an arbitrary file to > > > >> trigger the reload but got to the point of looking at the API so > > > >> figured I would send out a list message for any ideas/concerns/etc > > > >> > > > >> - Evan > > > > > > > > > > > > > > > > -- > > > > John Rushford > > > > [email protected] > > > >
