Re: How make your module substitute a python stdlib module.
On 12/28/2022 5:39 AM, Antoon Pardon wrote: Op 27/12/2022 om 16:49 schreef Thomas Passin: On 12/27/2022 8:25 AM, Antoon Pardon wrote: Op 27/12/2022 om 13:46 schreef Chris Angelico: On Tue, 27 Dec 2022 at 23:28, Antoon Pardon wrote: At the moment I am happy with a solution that once the programmer has imported from QYZlib.threaders that module will used as the threading module. Oh! If that's all you need, then yes, a simple patch of sys.modules will work. You'll have to make sure you get your module imported before anything else pulls up the vanilla one, though. Otherwise, your only option is to keep the existing module object and monkeypatch it with whatever changes you need. But a simple "sys.modules['threading'] = QYZlib.threaders" will work. Of course, how *well* this works depends also on how well that module manages to masquerade as the threading module, but I'm sure you've figured that part out :) Well it is what will work for the moment. Thanks for the confirmation this will indeed work. What you **should not** do is to make other modules use your code when they think they are using the standard library. Raymond Chen in The Old New Thing blog often writes about users who want to do a basically similar thing. He always asks "What if some other program wants to do the same thing?" One case was when a developer wanted his program's icon to force itself to be at the top position in the Windows task bar. "What if another program tried this too?" The two would keep trying to displace each other at the top position. If you set the PYTHONPATH environmental variable, you can put any directory you like to be at the start of the module search path after the current directory. Perhaps this would be enough for your use case? But if I put a threading.py file in that directory, it would make other modules use my code when they "think" they are using the standard library. The problem is that the logging module uses threading, but doesn't allow you to specify which threading module to use. So if you want to use an alternative, you either have to do things like this, or copy the logging packages into your project and adapt it to use your alternative. In that case, I think your best bet would be to write wrappers for the threading classes or functions you want to use. Your wrappers would import from the threading module and could then produce the behavior you want. -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Op 27/12/2022 om 16:49 schreef Thomas Passin: On 12/27/2022 8:25 AM, Antoon Pardon wrote: Op 27/12/2022 om 13:46 schreef Chris Angelico: On Tue, 27 Dec 2022 at 23:28, Antoon Pardon wrote: At the moment I am happy with a solution that once the programmer has imported from QYZlib.threaders that module will used as the threading module. Oh! If that's all you need, then yes, a simple patch of sys.modules will work. You'll have to make sure you get your module imported before anything else pulls up the vanilla one, though. Otherwise, your only option is to keep the existing module object and monkeypatch it with whatever changes you need. But a simple "sys.modules['threading'] = QYZlib.threaders" will work. Of course, how *well* this works depends also on how well that module manages to masquerade as the threading module, but I'm sure you've figured that part out :) Well it is what will work for the moment. Thanks for the confirmation this will indeed work. What you **should not** do is to make other modules use your code when they think they are using the standard library. Raymond Chen in The Old New Thing blog often writes about users who want to do a basically similar thing. He always asks "What if some other program wants to do the same thing?" One case was when a developer wanted his program's icon to force itself to be at the top position in the Windows task bar. "What if another program tried this too?" The two would keep trying to displace each other at the top position. If you set the PYTHONPATH environmental variable, you can put any directory you like to be at the start of the module search path after the current directory. Perhaps this would be enough for your use case? But if I put a threading.py file in that directory, it would make other modules use my code when they "think" they are using the standard library. The problem is that the logging module uses threading, but doesn't allow you to specify which threading module to use. So if you want to use an alternative, you either have to do things like this, or copy the logging packages into your project and adapt it to use your alternative. -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
On 12/27/2022 8:25 AM, Antoon Pardon wrote: Op 27/12/2022 om 13:46 schreef Chris Angelico: On Tue, 27 Dec 2022 at 23:28, Antoon Pardon wrote: At the moment I am happy with a solution that once the programmer has imported from QYZlib.threaders that module will used as the threading module. Oh! If that's all you need, then yes, a simple patch of sys.modules will work. You'll have to make sure you get your module imported before anything else pulls up the vanilla one, though. Otherwise, your only option is to keep the existing module object and monkeypatch it with whatever changes you need. But a simple "sys.modules['threading'] = QYZlib.threaders" will work. Of course, how *well* this works depends also on how well that module manages to masquerade as the threading module, but I'm sure you've figured that part out :) Well it is what will work for the moment. Thanks for the confirmation this will indeed work. What you **should not** do is to make other modules use your code when they think they are using the standard library. Raymond Chen in The Old New Thing blog often writes about users who want to do a basically similar thing. He always asks "What if some other program wants to do the same thing?" One case was when a developer wanted his program's icon to force itself to be at the top position in the Windows task bar. "What if another program tried this too?" The two would keep trying to displace each other at the top position. If you set the PYTHONPATH environmental variable, you can put any directory you like to be at the start of the module search path after the current directory. Perhaps this would be enough for your use case? -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Antoon Pardon wrote at 2022-12-27 14:25 +0100: > ... >> But a simple "sys.modules['threading'] = QYZlib.threaders" will work. >> Of course, how *well* this works depends also on how well that module >> manages to masquerade as the threading module, but I'm sure you've >> figured that part out :) > >Well it is what will work for the moment. Thanks for the confirmation >this will indeed work. If you need to change a module in minor ways (e.g. only provide a custom `thread_ident` function), you can use a technique called "monkey patching" (which is patching at runtime). You can usually assign new values to module variables. Thus, you yould try `threading.thread_ident = `. This would affect most uses of the function -- which may not be a good idea. Alternatively, you could monkey patch the `logging` module. Look at its code and find out whether it accesses the function directly or indirectly via `threading`. In the first case, you would monkey patch the function, in the second the `threading` variable. You could also use `dm.reuse` (a package maintained on PyPI) to override the method using the function. This way, your change would be even more localized. -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Antoon Pardon wrote: Op 27/12/2022 om 11:37 schreef Chris Angelico: On Tue, 27 Dec 2022 at 21:29, Antoon Pardon wrote: OK, I am writing an alternative for the threading module. What I would like to know is how I can get some library modules call my alternative instead of the threading module. For instance there is the logging module, it can log the thread name. So I would like to know how I can get the logging module to call the function from my module to get the current_thread, instead of it calling "current_thread" from the threading module. Easy: make sure your module is called "threading.py" and is earlier in the path than the standard library. In fact, it's so easy that people do it unintentionally all the time... Generally, the current directory (or the script directory) is the first entry in sys.path, so that's a good place to put it. Well I had hope for a somewhat more selective solution. The intention is to have a number of modules collected in a package where this module is one of and the package is available via the "installed" search path. So the programmer should just be able to choose to write his code with either: from threading import Thread or from QYZlib.threaders import Thread In that case, it sounds like you don't want to always replace the logging module's behaviour, since code might use the standard threading module. You might also need to consider that a mix of both the standard threading module and yours might be used (particularly if using other libraries which create their own threads using the standard module). It might be better for your module to provide a custom `logging.Filter` subclass, which replaces the `thread` and `threadName` attributes of the `LogRecord` with your threading module's idea of the thread ID. Then applications using your threading module would configure the `logging` module to use your filter, while code using the standard threading module can continue as normal. Despite the name, logging filters can also modify log records, not just filter whether or not they're logged. It might even be desirable to leave the original `thread` and `threadName` attributes unchanged and add new attributes to the `LogRecord` for your module's idea of threads (e.g. `myThread` and `myThreadName`). I think using `%(myThread)d` and `%(myThreadName)s` in the log format string would use those attributes, without needing a custom formatter. That would allow both thread IDs to be logged, in case a mix of standard threads and your threads is used. -- Mark. -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Op 27/12/2022 om 13:46 schreef Chris Angelico: On Tue, 27 Dec 2022 at 23:28, Antoon Pardon wrote: At the moment I am happy with a solution that once the programmer has imported from QYZlib.threaders that module will used as the threading module. Oh! If that's all you need, then yes, a simple patch of sys.modules will work. You'll have to make sure you get your module imported before anything else pulls up the vanilla one, though. Otherwise, your only option is to keep the existing module object and monkeypatch it with whatever changes you need. But a simple "sys.modules['threading'] = QYZlib.threaders" will work. Of course, how *well* this works depends also on how well that module manages to masquerade as the threading module, but I'm sure you've figured that part out :) Well it is what will work for the moment. Thanks for the confirmation this will indeed work. -- Antoon Pardon. -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
On Tue, 27 Dec 2022 at 23:28, Antoon Pardon wrote: > > > > Op 27/12/2022 om 13:09 schreef Chris Angelico: > > On Tue, 27 Dec 2022 at 23:06, Antoon Pardon wrote: > >>> How do you intend to distinguish one from the other? How should the > >>> logging module know which threading module to use? > >> That is my question! How can I get the logging module to use my module.I > >> was hoping the logging module would allow some kind of dependency > >> injection, so you can tell it what threading module to use. An other > >> option might be to manipulate sys.modules. -- Antoon Pardon > > But presumably you want OTHER modules to continue using the vanilla > > threading module? This is likely to end up somewhat hacky. Yes, you > > can manipulate sys.modules, but there's only one threading module at a > > time, so you need a way to distinguish between modules that should use > > the changed one and modules that shouldn't. > > But don't these caveats also apply with your original answer of having a > local threading.py file? > > At the moment I am happy with a solution that once the programmer has > imported from QYZlib.threaders that module will used as the threading > module. > Oh! If that's all you need, then yes, a simple patch of sys.modules will work. You'll have to make sure you get your module imported before anything else pulls up the vanilla one, though. Otherwise, your only option is to keep the existing module object and monkeypatch it with whatever changes you need. But a simple "sys.modules['threading'] = QYZlib.threaders" will work. Of course, how *well* this works depends also on how well that module manages to masquerade as the threading module, but I'm sure you've figured that part out :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Op 27/12/2022 om 13:09 schreef Chris Angelico: On Tue, 27 Dec 2022 at 23:06, Antoon Pardon wrote: How do you intend to distinguish one from the other? How should the logging module know which threading module to use? That is my question! How can I get the logging module to use my module.I was hoping the logging module would allow some kind of dependency injection, so you can tell it what threading module to use. An other option might be to manipulate sys.modules. -- Antoon Pardon But presumably you want OTHER modules to continue using the vanilla threading module? This is likely to end up somewhat hacky. Yes, you can manipulate sys.modules, but there's only one threading module at a time, so you need a way to distinguish between modules that should use the changed one and modules that shouldn't. But don't these caveats also apply with your original answer of having a local threading.py file? At the moment I am happy with a solution that once the programmer has imported from QYZlib.threaders that module will used as the threading module. -- Antoon Pardon. -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
On Tue, 27 Dec 2022 at 23:06, Antoon Pardon wrote: > > How do you intend to distinguish one from the other? How should the > > logging module know which threading module to use? > > That is my question! How can I get the logging module to use my module.I was > hoping the logging module would allow some kind of dependency > injection, so you can tell it what threading module to use. An other > option might be to manipulate sys.modules. -- Antoon Pardon But presumably you want OTHER modules to continue using the vanilla threading module? This is likely to end up somewhat hacky. Yes, you can manipulate sys.modules, but there's only one threading module at a time, so you need a way to distinguish between modules that should use the changed one and modules that shouldn't. TBH this is starting to sound like something where you want unittest-type facilities, actually. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Op 27/12/2022 om 12:28 schreef Chris Angelico: On Tue, 27 Dec 2022 at 22:13, Antoon Pardon wrote: Op 27/12/2022 om 11:37 schreef Chris Angelico: On Tue, 27 Dec 2022 at 21:29, Antoon Pardon wrote: OK, I am writing an alternative for the threading module. What I would like to know is how I can get some library modules call my alternative instead of the threading module. For instance there is the logging module, it can log the thread name. So I would like to know how I can get the logging module to call the function from my module to get the current_thread, instead of it calling "current_thread" from the threading module. Easy: make sure your module is called "threading.py" and is earlier in the path than the standard library. In fact, it's so easy that people do it unintentionally all the time... Generally, the current directory (or the script directory) is the first entry in sys.path, so that's a good place to put it. Well I had hope for a somewhat more selective solution. The intention is to have a number of modules collected in a package where this module is one of and the package is available via the "installed" search path. So the programmer should just be able to choose to write his code with either: from threading import Thread or from QYZlib.threaders import Thread How do you intend to distinguish one from the other? How should the logging module know which threading module to use? That is my question! How can I get the logging module to use my module.I was hoping the logging module would allow some kind of dependency injection, so you can tell it what threading module to use. An other option might be to manipulate sys.modules. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
On Tue, 27 Dec 2022 at 22:13, Antoon Pardon wrote: > > > > Op 27/12/2022 om 11:37 schreef Chris Angelico: > > On Tue, 27 Dec 2022 at 21:29, Antoon Pardon wrote: > >> OK, I am writing an alternative for the threading module. What I would > >> like to know is how I can get some library modules call my alternative > >> instead of the threading module. > >> > >> For instance there is the logging module, it can log the thread name. So > >> I would like to know how I can get the logging module to call the > >> function from my module to get the current_thread, instead of it calling > >> "current_thread" from the threading module. > > Easy: make sure your module is called "threading.py" and is earlier in > > the path than the standard library. In fact, it's so easy that people > > do it unintentionally all the time... Generally, the current directory > > (or the script directory) is the first entry in sys.path, so that's a > > good place to put it. > > Well I had hope for a somewhat more selective solution. The intention is > to have a number of modules collected in a package where this module is > one of and the package is available via the "installed" search path. > > So the programmer should just be able to choose to write his code with > either: > > from threading import Thread > > or > > from QYZlib.threaders import Thread > How do you intend to distinguish one from the other? How should the logging module know which threading module to use? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
Op 27/12/2022 om 11:37 schreef Chris Angelico: On Tue, 27 Dec 2022 at 21:29, Antoon Pardon wrote: OK, I am writing an alternative for the threading module. What I would like to know is how I can get some library modules call my alternative instead of the threading module. For instance there is the logging module, it can log the thread name. So I would like to know how I can get the logging module to call the function from my module to get the current_thread, instead of it calling "current_thread" from the threading module. Easy: make sure your module is called "threading.py" and is earlier in the path than the standard library. In fact, it's so easy that people do it unintentionally all the time... Generally, the current directory (or the script directory) is the first entry in sys.path, so that's a good place to put it. Well I had hope for a somewhat more selective solution. The intention is to have a number of modules collected in a package where this module is one of and the package is available via the "installed" search path. So the programmer should just be able to choose to write his code with either: from threading import Thread or from QYZlib.threaders import Thread -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Re: How make your module substitute a python stdlib module.
On Tue, 27 Dec 2022 at 21:29, Antoon Pardon wrote: > > OK, I am writing an alternative for the threading module. What I would > like to know is how I can get some library modules call my alternative > instead of the threading module. > > For instance there is the logging module, it can log the thread name. So > I would like to know how I can get the logging module to call the > function from my module to get the current_thread, instead of it calling > "current_thread" from the threading module. Easy: make sure your module is called "threading.py" and is earlier in the path than the standard library. In fact, it's so easy that people do it unintentionally all the time... Generally, the current directory (or the script directory) is the first entry in sys.path, so that's a good place to put it. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
How make your module substitute a python stdlib module.
OK, I am writing an alternative for the threading module. What I would like to know is how I can get some library modules call my alternative instead of the threading module. For instance there is the logging module, it can log the thread name. So I would like to know how I can get the logging module to call the function from my module to get the current_thread, instead of it calling "current_thread" from the threading module. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
