Re: 'reload M' doesn't update 'from M inport *'
In article mailman.616.1278927754.1673.python-l...@python.org, Jean-Michel Pichavant jeanmic...@sequans.com wrote: Aahz wrote: In article mailman.488.1278697107.1673.python-l...@python.org, Jean-Michel Pichavant jeanmic...@sequans.com wrote: PS : You're misusing the del statement. It does not remove any object from mmory, however, it removes the reference to it, the object is still in memory. They are very few cases where del is usefull in python, so try to avoid using it as well. The first two sentences are true; the last sentence is completely wrong. I've got lots of code using del, and it's a critically useful element of dictionary manipulation. Can you please give a short example ? I'm not challenging though, just curious. Let's suppose you want to remove duplicate keys from a second dict: for k in d1: if k in d2: del d2[k] You could do d2.pop() instead, but I'll bet you'll find that it's slightly slower, which makes a difference in a tight loop. (Mainly, though, I learned this idiom long before pop() was available.) -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Normal is what cuts off your sixth finger and your tail... --Siobhan -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
Aahz wrote: In article mailman.488.1278697107.1673.python-l...@python.org, Jean-Michel Pichavant jeanmic...@sequans.com wrote: PS : You're misusing the del statement. It does not remove any object from mmory, however, it removes the reference to it, the object is still in memory. They are very few cases where del is usefull in python, so try to avoid using it as well. The first two sentences are true; the last sentence is completely wrong. I've got lots of code using del, and it's a critically useful element of dictionary manipulation. Can you please give a short example ? I'm not challenging though, just curious. JM -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
Hi, Don't use reload, this is nothing but a trap, espacially if your using it to update your objects with the code you are writting. JM I've found reload very usable for development in IDLE. IDLE memorizes my input, and the variables I assign output to. If restart IDLE I lose it all and start over. That is an awfully awkward alternative to reload, an alternative I wouldn't consider. I found reload tricky with several modules, because all dependencies need to be updated and which way they go isn't always obvious. Reloading all modules in the right order works for me. The reload commands come up with Alt-P as long, precisely, as I don't restart IDLE. Instead of writing your test code in the IDLE shell, write it into a file, an run this file in IDLE (from the IDLE file edit window hit F5). That way restarting from the begining is costless. If you insist on using reload, then you've already been given helpful answers, understanding the import model is always a good thing anyway :) JM -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
from m import f look for module m in the global cache if not there, then: search for m.py compile it to a Module object put the Module object in the cache look for object named f in the Module object agree create a new name f in the local namespace set the name f to cached object strongly agree The important thing to notice is the the name f is a local variable. It doesn't, and can't, remember that it comes from module m. Reloading m can't do anything to f, because the connection is lost. disagree with 2nd stmt. local 'f' does *remember* (if that's the right word) where it comes from, because it points to the original obj, as you said: 'set the name f to cached object' py0.py == def f(): ... from py0 import * assert f.__module__ == sys.modules['py0'].__name__ assert f.__globals__ is sys.modules['py0'].__dict__ Now consider that the object f that came from m was itself imported from another module, service. Reloading service doesn't help, because m.f doesn't know it came from service. Reloading m doesn't help, because all that does is run from service import f again, and that just fetches f from the global cache. disagree with 2nd stmt, partially disagree with 3rd stmt reloading 'service' partially helps, since it updates the mod obj pointed by 'service' in global cache. it needs to be followed by reloading m, then we have m.f points to the new obj. the important part is the order of reloading mods l.py def f(): ... m.py from l import * from m import * at this point, the func obj is referenced by 3 distinct variables with name 'm'(one in each mod) assert sys.getrefcount(f) == 4 referrers = gc.get_referrers(f) mod_dicts = [sys.modules[k].__dict__ for k in sys.modules if k == 'l' or k == 'm' or k == __name__] for d in mod_dicts: ... referrers.remove(d) assert len(referrers) == 0 imp.reload(sys.modules['l']) now the original func obj is ref'ed by 2 vars, the new func obj is ref'ed by 1 var imp.reload(sys.modules['m']) f = sys.modules['m'].f now the original func obj is ready to be recollected, the new func obj is ref'ed by 3 vars The simplest, easiest way of dealing with this is not to have to deal with it: don't use from service import f, and ESPECIALLY don't use from service import *. Always use fully-qualified importing: import service service.f strongly agree The other way is not to bother with reload. It's not very powerful, only good for the simplest use in the interactive interpreter. Just exit the interpreter and restart it. set, del, reload are useful when it comes to structure manipulation at runtime, most langs differentiate it, labeling as metaprogramming, py smashes the diff in an elegant way (oh flattery, i love it) ;) as their names say, set del only bind unbind obj to var, the obj to bind must be in its bytecode form, in theory one could do it, maybe thru modules that are categorized as Python Language Services in the lib manual, but it's not practical (e.g. see the last stmt of types.CodeType.__doc__). this is where 'reload' steps in, and so the story goes ... -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
Frederic Rentsch anthra.nor...@bluewin.ch wrote: I develop in an IDLE window. Module M says 'from service import *'. Next I correct a mistake in function 'service.f'. Now 'service.f' works fine. I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. 'print inspect.getsource (service.f)' and 'print inspect.getsource (M.f)' shows the same corrected code. 'print service.f' and 'print M.f' show different ids. Yes. This: from service import xxx is essentially the same as: import service xxx = service.xxx At that point, xxx contains a reference to the service.xxx object as it is right now. When you do a reload, that imports a new version of service.xxx, but your global xxx object is still bound to the old one. -- Tim Roberts, t...@probo.com Providenza Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
In article mailman.465.1278680555.1673.python-l...@python.org, Frederic Rentsch anthra.nor...@bluewin.ch wrote: Module M says 'from service import *'. Next I correct a mistake in function 'service.f'. Now 'service.f' works fine. I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. Absolutely! 'print inspect.getsource (service.f)' and 'print inspect.getsource (M.f)' shows the same corrected code. 'print service.f' and 'print M.f' show different ids. So I do 'del M; reload (M)'. Nothing changes. I delete M again and run gc.collect () to really clean house. I reload M again and still nothing changes. The id of the reloaded function 'M.f' is still the same as it was before the purge and so M.f still isn't fixed. I know I have more radical options, such as starting a new IDLE window. That would save me time, but I'd like to take the opportunity to understand what is happening. Surely someone out there knows. Take a look at sys.modules to get a better idea of what's happening. (Maybe someone else will have time to write a longer answer.) But really, relying on reload() is foolish in the general case because it's nearly impossible to track down every single reference. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Normal is what cuts off your sixth finger and your tail... --Siobhan -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
On Fri, 09 Jul 2010 15:02:25 +0200, Frederic Rentsch wrote: I develop in an IDLE window. Module M says 'from service import *'. Next I correct a mistake in function 'service.f'. Now 'service.f' works fine. from service import * should be considered advanced functionality that is discouraged unless you really know what you are doing, precisely for the problems you are experiencing. You should try to avoid it. But putting that aside, if you have done from service import * in module m, where are you getting service.f from? The only way that is possible is if you ALSO say import service. I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. 'print inspect.getsource (service.f)' and 'print inspect.getsource (M.f)' shows the same corrected code. inspect.getsource always looks at the source code on disk, no matter what the byte code in memory actually says. 'print service.f' and 'print M.f' show different ids. So I do 'del M; reload (M)'. Nothing changes. I delete M again and run gc.collect () to really clean house. I reload M again and still nothing changes. The id of the reloaded function 'M.f' is still the same as it was before the purge and so M.f still isn't fixed. I know I have more radical options, such as starting a new IDLE window. That would save me time, but I'd like to take the opportunity to understand what is happening. Surely someone out there knows. Yes. You have to understand importing. Let's start with the simple: import m In *very* simplified pseudo-code, this does: look for module m in the global cache if not there, then: search for m.py compile it to a Module object put the Module object in the cache create a new name m in the local namespace set the name m to the Module object in the cache Now let's compare it to: from m import f look for module m in the global cache if not there, then: search for m.py compile it to a Module object put the Module object in the cache look for object named f in the Module object create a new name f in the local namespace set the name f to cached object The important thing to notice is the the name f is a local variable. It doesn't, and can't, remember that it comes from module m. Reloading m can't do anything to f, because the connection is lost. Now consider that the object f that came from m was itself imported from another module, service. Reloading service doesn't help, because m.f doesn't know it came from service. Reloading m doesn't help, because all that does is run from service import f again, and that just fetches f from the global cache. The simplest, easiest way of dealing with this is not to have to deal with it: don't use from service import f, and ESPECIALLY don't use from service import *. Always use fully-qualified importing: import service service.f Now reload(service) should do what you expect. The other way is not to bother with reload. It's not very powerful, only good for the simplest use in the interactive interpreter. Just exit the interpreter and restart it. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
I know I have more radical options, such as starting a new IDLE window. That would save me time, but I'd like to take the opportunity to understand what is happening. Surely someone out there knows. Frederic Or you can restart the IDLE shell with CTRL+F6. If you can't restart it, you're probably not using an IDLE subprocess: On linux, make sure the menu item doesn't use -n in the command On Windows, first open IDLE, then open the file, or change the Open command to not use the -n flag. Sorry to not chip in to your question, frankly, Steven nailed it! Just thought to share an IDLE tip :) Nick -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
Frederic Rentsch wrote: I develop in an IDLE window. Module M says 'from service import *'. Next I correct a mistake in function 'service.f'. Now 'service.f' works fine. I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. 'print inspect.getsource (service.f)' and 'print inspect.getsource (M.f)' shows the same corrected code. 'print service.f' and 'print M.f' show different ids. So I do 'del M; reload (M)'. Nothing changes. I delete M again and run gc.collect () to really clean house. I reload M again and still nothing changes. The id of the reloaded function 'M.f' is still the same as it was before the purge and so M.f still isn't fixed. I know I have more radical options, such as starting a new IDLE window. That would save me time, but I'd like to take the opportunity to understand what is happening. Surely someone out there knows. Frederic Hi, Don't use reload, this is nothing but a trap, espacially if your using it to update your objects with the code you are writting. JM PS : You're misusing the del statement. It does not remove any object from mmory, however, it removes the reference to it, the object is still in memory. They are very few cases where del is usefull in python, so try to avoid using it as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
On Fri, 2010-07-09 at 15:58 +, Steven D'Aprano wrote: On Fri, 09 Jul 2010 15:02:25 +0200, Frederic Rentsch wrote: I develop in an IDLE window. Module M says 'from service import *'. Next I correct a mistake in function 'service.f'. Now 'service.f' works fine. from service import * should be considered advanced functionality that is discouraged unless you really know what you are doing, precisely for the problems you are experiencing. You should try to avoid it. But putting that aside, if you have done from service import * in module m, where are you getting service.f from? The only way that is possible is if you ALSO say import service. I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. 'print inspect.getsource (service.f)' and 'print inspect.getsource (M.f)' shows the same corrected code. inspect.getsource always looks at the source code on disk, no matter what the byte code in memory actually says. 'print service.f' and 'print M.f' show different ids. So I do 'del M; reload (M)'. Nothing changes. I delete M again and run gc.collect () to really clean house. I reload M again and still nothing changes. The id of the reloaded function 'M.f' is still the same as it was before the purge and so M.f still isn't fixed. I know I have more radical options, such as starting a new IDLE window. That would save me time, but I'd like to take the opportunity to understand what is happening. Surely someone out there knows. Yes. You have to understand importing. Let's start with the simple: import m In *very* simplified pseudo-code, this does: look for module m in the global cache if not there, then: search for m.py compile it to a Module object put the Module object in the cache create a new name m in the local namespace set the name m to the Module object in the cache Now let's compare it to: from m import f look for module m in the global cache if not there, then: search for m.py compile it to a Module object put the Module object in the cache look for object named f in the Module object create a new name f in the local namespace set the name f to cached object The important thing to notice is the the name f is a local variable. It doesn't, and can't, remember that it comes from module m. Reloading m can't do anything to f, because the connection is lost. Now consider that the object f that came from m was itself imported from another module, service. Reloading service doesn't help, because m.f doesn't know it came from service. Reloading m doesn't help, because all that does is run from service import f again, and that just fetches f from the global cache. The simplest, easiest way of dealing with this is not to have to deal with it: don't use from service import f, and ESPECIALLY don't use from service import *. Always use fully-qualified importing: import service service.f Now reload(service) should do what you expect. The other way is not to bother with reload. It's not very powerful, only good for the simplest use in the interactive interpreter. Just exit the interpreter and restart it. -- Steven Thank you very much for your excellent explanation! I must say that I haven't been using the from soandso import ... formula at all. I thought it might expose names to collision, and why should I assume the responsibility if I can avoid the problem altogether using explicit names. If I used the, shall we say, direct import this time it was in an effort to develop a more extensive program. I thought if a module grows beyond a size that's comfortable to edit, I could just move select segments to separate files and replace the vacancy with from the_respective_segment_module import *, analogous to #include in C. The remedy seems to have side-effects that can kill the patient. So I'll go back to the explicit imports, then. No problem at all. Thanking you and the other helpers too Frederic -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
In article mailman.488.1278697107.1673.python-l...@python.org, Jean-Michel Pichavant jeanmic...@sequans.com wrote: PS : You're misusing the del statement. It does not remove any object from mmory, however, it removes the reference to it, the object is still in memory. They are very few cases where del is usefull in python, so try to avoid using it as well. The first two sentences are true; the last sentence is completely wrong. I've got lots of code using del, and it's a critically useful element of dictionary manipulation. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Normal is what cuts off your sixth finger and your tail... --Siobhan -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
On Fri, 2010-07-09 at 19:38 +0200, Jean-Michel Pichavant wrote: Frederic Rentsch wrote: I develop in an IDLE window. Module M says 'from service import *'. Next I correct a mistake in function 'service.f'. Now 'service.f' works fine. I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. 'print inspect.getsource (service.f)' and 'print inspect.getsource (M.f)' shows the same corrected code. 'print service.f' and 'print M.f' show different ids. So I do 'del M; reload (M)'. Nothing changes. I delete M again and run gc.collect () to really clean house. I reload M again and still nothing changes. The id of the reloaded function 'M.f' is still the same as it was before the purge and so M.f still isn't fixed. I know I have more radical options, such as starting a new IDLE window. That would save me time, but I'd like to take the opportunity to understand what is happening. Surely someone out there knows. Frederic Hi, Don't use reload, this is nothing but a trap, espacially if your using it to update your objects with the code you are writting. JM I've found reload very usable for development in IDLE. IDLE memorizes my input, and the variables I assign output to. If restart IDLE I lose it all and start over. That is an awfully awkward alternative to reload, an alternative I wouldn't consider. I found reload tricky with several modules, because all dependencies need to be updated and which way they go isn't always obvious. Reloading all modules in the right order works for me. The reload commands come up with Alt-P as long, precisely, as I don't restart IDLE. S : You're misusing the del statement. It does not remove any object from mmory, however, it removes the reference to it, the object is still in memory. They are very few cases where del is usefull in python, so try to avoid using it as well. I understand that things going out of scope delete themselves. I have used del on occasion, for instance, to get rid of invalid members of a list or a dictionary. It has to be done in two passes, though, because neither can be altered during an iteration. The first pass makes a delete list of indices or keys, so that the actual deletion iterates through the delete list, not the object deleted from. Would you call that a misuse? Frederic -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
On 7/9/2010 9:02 AM, Frederic Rentsch wrote: I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. Guido removed reload from 3.0 because it gave people false hopes and he gave up on fixing it. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
On Fri, 09 Jul 2010 17:46:26 -0400, Terry Reedy wrote: On 7/9/2010 9:02 AM, Frederic Rentsch wrote: I do 'reload (service); reload (M)'. The function 'M.f' still misbehaves. Guido removed reload from 3.0 because it gave people false hopes and he gave up on fixing it. It's not quite *removed*. It's just no longer a built-in: import imp imp.reload(spam) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: 'reload M' doesn't update 'from M inport *'
On Fri, 09 Jul 2010 23:14:23 +0200, Frederic Rentsch wrote: I understand that things going out of scope delete themselves. I have used del on occasion, for instance, to get rid of invalid members of a list or a dictionary. It has to be done in two passes, though, because neither can be altered during an iteration. The first pass makes a delete list of indices or keys, so that the actual deletion iterates through the delete list, not the object deleted from. Would you call that a misuse? No, that is perfectly fine. Pay no attention to anyone who says that del should be avoided. Like all tools, it can be used badly or inefficiently, or by somebody who doesn't understand it, but that's no reason to avoid using del correctly. Another strategy is to make a copy of the dict/list and iterate over that. Copies are fast, since you're only copying the top-level list or dict and not the underlying objects within it. A third is to forget about deleting and create a new list containing only the objects you don't want to delete. All three strategies (two using del, one without) have different use-cases and are perfectly acceptable. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
