One of the issues I've found with loading classes eagerly is that many ClassLoader implementations rely on fairly broad locks. Deferred class loading can avoid some of this lock contention.
On Thu, Apr 7, 2022 at 12:57 PM Ralph Goers <ralph.go...@dslextreme.com> wrote: > > > > > On Apr 7, 2022, at 2:52 AM, Peter Verhas <pe...@verhas.com> wrote: > > > > > >> I would suggest that before responding to this email that > >> you go look at how log4j-plugins is implemented in the master branch. > >> > > > > Sure, if you propose to create a commons-plugin library that is an extract > > of the plugin handling of log4j so that log4j next releases do not need to > > keep the functionality inside log4j but rather as a dependency on this new > > library. On the other hand, if the goal is to create something that is > > useful generally then we should look at other solutions as well. > > It remains to be seen whether Log4j would replace log4j-plugins with > Commons-plugins. The point of looking at Log4j was to understand the > problem it is solving and how it solves it. > > > > > > > >> From this it is clear that you completely misunderstand what the plugin > >> system is doing. > >> > > > > That may really be very much true. > > > > > >> First, the plugin system DOES use ServiceLoader. In fact, Log4j uses > >> ServiceLoader > >> in at least 7 different places. For one, it uses ServiceLoader to locate > >> the Log4j > >> implementation similar to how SLF4J does (although Log4j implemented it > >> prior to SLF4J). > >> > >> Consider that you have 150 or so plugins, of which maybe only 10 will get > >> used. But > >> you don’t know beforehand which 10. ServiceLoader would have to load all > >> 150 classes. > > > > > > Not. It does not need to create instances of all of them. This is what I > > explained, and this is the reason why I explained it. JPMS or not, the new > > ServiceLoader used by Java 9 and later will call the static factory method > > of the class if that exists. Then the plugin can decide if it wants itself > > to be instantiated or not. Somewhere there should be some logic that > > filters 10 from the 150. If that is in a central place implementing some > > industry-best-practice configuration based algorithm, or outsources the > > decision to the plugins themselves should not make a big difference in > > performance. > > > > On the second though, instantiating 150 lightweight classes must not be a > > big burden. The ServiceProvider JavaDoc documentation suggest that services > > should be implemented as proxies or factories instantiating their real > > working class lazily in case the instantiation is costly. > > Class loading in general is slow. Perhaps not to you but we have customers who > complain about all the things we do during startup and locating and > instantiating > plugins is one of them. When we first switched to use the ServiceLoader we > attempted to load the plugin classes simply by having the plugin definition > include > the Class.Performance was very bad. We now only include the class name and > only access the Class when absolutely necessary. > > Again, if you think about what the Plugin Manager is doing, it is using > ServiceLoader > to find all Plugin Service instances. Each of these contains the definitions > of all the > plugins. The Plugin Manager then uses these to create the actual plugins. So > in > essence, the services are used as part of the factory. Note that decisions > about > whether to load a plugin or not do not have to be made when the plugin service > s accessed. That is deferred until the Plugin actually needs to be wired. > > > > > > >> This is indeed very slow. Yes we have benchmarked it. But the Plugin > >> system IS a > >> Java Service. So instead, you need to only load 1 class for every jar that > >> contains plugins. > >> The result is that you only need to load 4 or 5 classes. This is > >> considerably faster. > >> > > > > Yes, this is the approach where you insert another layer above the service > > loader that does part of the job that the service loader can also do in a > > different way. Essentially you let the plugin libraries, each containing > > several plugin classes to implement their own service loaders. > > Sort of. With Log4j’s plugin system Plugins are not aware of ServiceLoader > at all. Other than that, it behaves the way you are describing. > > > > > > >> In addition, services loaded by ServiceLoader are singletons. > > > > > > No, they are not. Every ServiceLoader instance will create new instances, > > and even a single ServiceLoader can be asked calling the `reload()` method > > to evict the instances from its cache and create new instances. > > Umm. Even calling reload() doesn’t change it from being a singleton. You can > only have a single instance at a time. > > Yes, I suppose if you use multiple ServiceLoader instances then you can I > nstantiate multiple instances of the service. However, as you noted that > wouldn’t > be the best practice since the ServiceLoader really should be instantiating a > factory that create as many instances as you want. > > > > > Perhaps developing a commons-plugin system is a good point to require > > application developers and plugin developers to use the module system. > > Please no. I would never encourage anyone to use it. It has caused nothing > but pain. Yes, that is opinion, but opinion based on experience. > > > > > > >> Again, I would suggest you look at the log4j-plugins implementation before > >> making sweeping statements such as this. > >> > > > > I am not an expert about log4j and the plugin system of it,which you are. > > From this distance it seems that log4j uses factories for the plugins as > > services or builders in case there is a need for some configuration. In > > case of the builders the configuration parameters are injected that way > > this plugin system implements a simple DI framework like Plexus, Pico, > > Guava, Spring instead of using one. With the factory approach it does the > > same as the JDK ServiceLoader calling the `provider()` method. > > > > Again, if you find that answering the questions is difficult there may be > > several reasons. One thing is for sure: I am not trolling you. I am asking > > these questions because I honestly believe that any new library should > > address the things that are not addressed by other systems and libraries, > > and should provide a better way to do things, otherwise they will not be > > widely used. > > > Frankly, I don’t think there is more to discuss. You asked why not just use > ServiceLoader. But you have pretty much conceded that vanilla ServiceLoader > by itself is not a good plugin system. You have agreed that using > ServiceLoader > to load factories is pretty much necessary. But since you don’t want to look > at > he Log4j plugin system to see how it makes creating plugins easy it is hard to > discuss how it achieves the goals beyond just using ServiceLoader to load > factories. > > Ralph > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
