[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
Mike Kaplinskiy added the comment: Hey Nick, Definitely agree that this refactor is big enough to try adding target modules. There's a somewhat hidden feature in the second patch that does this: `use_globals_from_sys_modules` takes `sys.globals` from the `sys.modules` entry for the module. It's a constructor arg though. It's only used by `_run_module_as_main`, but making it more official sounds good. Given that goal, I'm a bit worried about how to accurately describe the behavior of `runnable.globals`. Particularly, what's a good API? Here are a couple of options I'm thinking of: - `x = load_module(...); x.module = sys.modules['foo']; x.run()`. Pros: allows setting the module that's to be used for maximal customizability. Cons: `x.globals` is poorly defined in that scenario. What should it reflect AFTER calling `x.run()`? What should it be before/after setting `x.module`? Should it overwrite __file__, __loader__, etc in the target module? Should it restore the values? When should it do this? - `x = load_module(...); x.overwrite_sys_modules = True; x.run()` This is a version of the above that perhaps makes it a bit easier to document what happens to which globals when. - `x = load_module(..., target_module=sys.modules['foo']); x.run()` Pros: less ambiguity about globals. They're always either local or the module's. Cons: all other "customizations" can be set after load_module is called. This is very asymmetric from an API perspective. There's also some ambiguity about what happens to the __file__, __line__, etc. - `x = load_module(...); x.run(target_module=...);` This is pretty much the sum of the cons of the above and more. Mostly here for completeness. I'm leaning towards the second option for API symmetry. The largest hurdle is defining a behavior w.r.t. globals that is least surprising. Maybe something like - if set to True, the globals in the target will be overwritten (i.e. .update) with the globals in the runner when `run()` is called. If folks want to save/restore anything around globals in the target module, they are free to do so themselves before calling .run(). Separately, what needs this type of behavior, other than for backwards compatibility? Do you know of any specific use-case? It feels like almost everything should be covered by a combination of add_to_sys_modules (i.e. temporary modules in sys.modules) and inspecting runner.globals after execution. What do you think? Mike. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
Mike Kaplinskiy added the comment: Hey Nick, Sorry for the long delay. Unfortunately Python isn't my main work language anymore so working on this has proved to be quite a context switch. I'm going to try to finish this up now. The attached patch implements a new pattern for wrapping runpy - one that I hope is a bit more general than just setting argv. In particular, using the new load_module/load_path doesn't automatically change argv at all when calling run. The callers can do pretty much whatever they want before calling run(). >From a docs perspective this is quite a bit easies to understand. You call >runpy.load_* to find the module, change whatever you want: globals/module dict >values/names/etc, and call .run(). We would even expose a convenient `with >runpy.ModifiedArgv(argv):` to help with the argv swapping. "Simple" use-cases >can continue to use runpy.run_* without needing to get into the save/restore >we do here. Let me know what you think of this approach and I can flesh out the docs around it. Otherwise, I'm more than happy to implement the callback approach you suggested. Thanks, Mike. -- Added file: http://bugs.python.org/file45201/patch-2.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
Mike Kaplinskiy added the comment: ping -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
Mike Kaplinskiy added the comment: So how might I get this patch committed? :) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
Mike Kaplinskiy added the comment: Looks like by signed CLA just made it through, so that should be settled. For the other bugs, it seems that overloading run_module & run_path seems to be getting a bit cumbersome, so it might make sense to have some sort of Runner object that has things like `Runner(module=..., code=..., path=...)` and then has properties like `argv = ...`, `module_dict = ...` and maybe attributes for the loader and the filepath. "Advanced" usage can use that over the vanilla run_module & run_path. Either way, I think that's outside the scope of this change. Unfortunately this is already shaving a yak for me (https://github.com/facebook/buck/pull/651#issuecomment-185030156) and I'd rather not go deeper. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
Changes by Mike Kaplinskiy <mike.kaplins...@gmail.com>: -- keywords: +patch Added file: http://bugs.python.org/file41981/patch.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26388] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
New submission from Mike Kaplinskiy: For the purposes of pex (https://github.com/pantsbuild/pex), it would be useful to allow calling run_module without sys.argv[0] changing. In general, this behavior is useful if the script intends to re-exec itself (so it needs to know the original arguments that it was started with). To make run_module more useful in general, I propose adding a `argv` parameter that has the following semantics: - (default) If set to a special value runpy.INHERIT (similar to subprocess.STDOUT), produces the current behavior of just changing sys.argv[0]. - If set to None, does not touch sys.argv at all. - If set to an iterable, executes `sys.argv = [module.__file__] + list(argv)`, and properly reverts it once the module is done executing. -- components: Library (Lib) messages: 260483 nosy: Mike Kaplinskiy priority: normal severity: normal status: open title: Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True) type: enhancement versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
