I'm not so sure about that. The author of the plugin knows they're writing code that will be dynamically loaded, and can therefore expect the kind of problem they're having. It could be argued that it's their responsibility to ensure that all the needed code is loaded into the subprocess.
Yes but I try to always make my libs/programs as much as usable as possible. "Ergonomic" would be the word. In the case of the plugin-engine I'm trying to hide any side-effect or unexpected behaviour of the engine so the developer of the plugin does not have take that into account. I agree that if the developer uses multiprocessing he/she needs to know its implications. But if I can "smooth" any rough corner, I will try to do it. For example, the main project (developed by me) uses threads for concurrency. It would be simpler to load the plugins and instantiate them *once* and ask the plugins developers to take care of any race condition (RC) within their implementation. Because the plugins were instantiated *once*, it is almost guaranteed that the plugins will suffer from race conditions and they will require some sort of locking. This is quite risky: you may forget to protect something and you will end up with a RC and/or you may put the lock in the wrong place and the whole thing will not work concurrently. My decision back then was to instantiate each plugin N+1 times: once in the main thread and then once per worker thread. With this, no single plugin instance will be shared so there is no risk of RC and no need for locking. (Yes, I know, the developer just needs to use a module variable or a class attribute and it will get a RC and these are shared but it is definitely not the default scenario). If sharing is required I provide an object that minimizes the locking needed. It was much complex for me at the design and at the implementation level but I think that it is safer and requires less from the plugin developer. Reference: https://byexamples.github.io/byexample/contrib/concurrency-model -- https://mail.python.org/mailman/listinfo/python-list