Martin Di Paola wrote at 2022-3-6 12:42 +0000: >Hi everyone. I implemented time ago a small plugin engine to load code >dynamically. > >So far it worked well but a few days ago an user told me that he wasn't >able to run in parallel a piece of code in MacOS. > >He was using multiprocessing.Process to run the code and in MacOS, the >default start method for such process is using "spawn". My understanding >is that Python spawns an independent Python server (the child) which >receives what to execute (the target function) from the parent process. > >In pseudo code this would be like: > >modules = loader() # load the plugins (Python modules at the end) >objs = init(modules) # initialize the plugins > ># One of the plugins wants to execute part of its code in parallel ># In MacOS this fails >ch = multiprocessing.Process(target=objs[0].sayhi) >ch.start() > >The code fails with "ModuleNotFoundError: No module named 'foo'" (where >'foo' is the name of the loaded plugin). > >This is because the parent program sends to the serve (the child) what >needs to execute (objs[0].sayhi) using pickle as the serialization >mechanism. > >Because Python does not really serialize code but only enough >information to reload it, the serialization of "objs[0].sayhi" just >points to its module, "foo". > >Module which it cannot be imported by the child process. > >So the question is, what would be the alternatives and workarounds?
Try to use `fork` as "start method" (instead of "spawn"). -- https://mail.python.org/mailman/listinfo/python-list