Re: [Tutor] Which is better Practice and why
On 23/10/12 12:53, Devin Jeanpierre wrote: On Mon, Oct 22, 2012 at 8:54 PM, Steven D'Apranost...@pearwood.info wrote: package.__main__.py is designed to be run as a script, and not to be imported. But that doesn't mean that there's no good purpose for importing it. If your package is non-trivial, you ought to have tests for it, including package.__main__. Those tests will probably want to import the module, not necessarily run it as a script. I think you will find that it is never the case that __name__ != '__main__' in a file called __main__.py. I think you'll find that you're wrong there :) [steve@ando ~]$ mkdir package [steve@ando ~]$ touch package/__init__.py [steve@ando ~]$ echo print(__name__) package/__main__.py [steve@ando ~]$ python package __main__ [steve@ando ~]$ python -c import package.__main__ package.__main__ If I want to test something, I put it in another file. Yes. And how do you import the __main__.py module to test it, without it executing? My __main__.py file will generally look something like this: from . import game game.Game().run() Well, that's probably trivial enough that it requires no testing other than does __main__.py exist?. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Tue, Oct 23, 2012 at 5:59 AM, Steven D'Aprano st...@pearwood.info wrote: I think you'll find that you're wrong there :) Oops. If I want to test something, I put it in another file. Yes. And how do you import the __main__.py module to test it, without it executing? How do you test the body of code in the if __name__ == '__main__' block, without it executing? -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 23 October 2012 02:17, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Mon, Oct 22, 2012 at 8:42 PM, Steven D'Aprano st...@pearwood.info wrote: If you do that, and the module directly or indirectly imports itself while it is running as a script, you may run into trouble. But writing a dual-purpose module that is usable as an importable module or as a stand-alone script is not problematic in itself. Yes. However, it is somewhat hard to guarantee that a module won't indirectly import itself for a large codebase. It certainly sometimes happens by accident. I guess this is the crux of the issue. If your script is part of and depends on a large codebase then you may as well place all of its code elsewhere in the codebase. The script then serves simply as an entry point and its presence in the project won't hinder your ability to move all the code it uses around to suit its relationship to the rest of the codebase. Importable scripts are useful when you have a codebase that is smaller. In particular if your codebase is one single script (a common case) then you should always use if __name__ == __main__ unless the script is trivial and/or, as Steven says, you're feeling lazy. Doing this means that you can test your script and its functions by importing the script in the interactive interpreter and that you can reuse the script as a module in some other project If your codebase is a single module rather than a script, giving it an if __name__ == __main__ block allows someone to perform a common task with the module (or perhaps run its tests) using the -m interpreter option without needing to turn a single file project into a double file project: $ python -m mymodule arg1 arg2 This is often better than creating a separate script to serve as a redundant entry point (redundant since the module can already serve as its own entry point). Once you have one importable script A.py you can then place another script B.py in the same folder have B import and use some of A's code. As Eryksun has said making this work both ways (having B also import from A) can lead to problems. If you find yourself wanting to do that then you have probably organised your code badly. As a solution consider moving all of the common code into one of A or B or into a new module/script C. Another factor not mentioned yet is that the use of if __name__ == __main__ is so ubiquitous that using it in your own script communicates something to most people who read it (including yourself). If I know that X.py is intended to be used as a script and I want to read it to find out how it works, one of the first things I would do is search for that line. My natural assumption is that no non-trivial execution occurs outside of that if-block (this is almost always true if I wrote the script and it is longer than about 10 lines). Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 7:45 AM, Matthew Ngaha chigga...@gmail.com wrote: the 2nd one usually includes a lot more code then i showed. can you please tell me why different methods are used to access the main code? is it just preference or is one way actually better coding practice? The second one is used if your importable modules are also scripts that can be executed. That's a bad idea, because it results in the same source file being executed twice, producing distinct classes that break typechecking. If you have myprogram.py containing the following source code, and execute it as a program, it will terminate with an uncaught MyError. class MyError(Exception): pass import myprogram try: raise myprogram.MyError except MyError: pass So, in any circumstance where you would use the second one, it's because you're in a situation where bad things are happening. So use the first one always. And if there's code in the script you want to share, put it in a separate module. -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 22/10/2012 12:45, Matthew Ngaha wrote: In many of the tutorial examples ive come across, the main code's program is never at the top level, but always in a function of some sort. i understand why but, there is always a different way used to access the main code, i want to know which is the best. main() main's code #top level main() they call the main program by simply calling the main function. I've also seen a more complcated: if __name__ == '__main__': main() the 2nd one usually includes a lot more code then i showed. can you please tell me why different methods are used to access the main code? is it just preference or is one way actually better coding practice? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor See this http://docs.python.org/tutorial/modules.html specifically section 6.1.1. Executing modules as scripts. -- Cheers. Mark Lawrence. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 7:45 AM, Matthew Ngaha chigga...@gmail.com wrote: In many of the tutorial examples ive come across, the main code's program is never at the top level, but always in a function of some sort. i understand why but, there is always a different way used to access the main code, i want to know which is the best. main() main's code #top level main() they call the main program by simply calling the main function. I've also seen a more complcated: if __name__ == '__main__': main() Every module in python has a name. You have probably seen code like this: import os import sys os and sys are modules and their names are 'os' and 'sys' When you write your own program in a file 'my_code.py' and then run it by typing: python my_code.py The name of your program is '__main__' So when you write: if __name__ == '__main__': your code here The code will only run when you run the file directly. Many times, the functions in your file could be used in other programs. In that case you might have a file called my_new_code.py In my_new_code.py you could import the other file so you could use its functions. Like this: import my_code When you import the file, it is not the __main__ code. Its name will be my_code. And so the code you have in the part I marked your code here will never run. It only runs when you run the file directly. -- Joel Goldstick ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
oh ok i understand it.. Thanks for the help guys ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 7:54 AM, Devin Jeanpierre jeanpierr...@gmail.com wrote: The second one is used if your importable modules are also scripts that can be executed. That's a bad idea, because it results in the same source file being executed twice, producing distinct classes that break typechecking. Here's an exception to the above. On Windows, multiprocessing requires you to gate the setup in __main__ since the module is re-imported in each new process (win32 doesn't fork). ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
Devin, On 22 October 2012 12:54, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Mon, Oct 22, 2012 at 7:45 AM, Matthew Ngaha chigga...@gmail.com wrote: the 2nd one usually includes a lot more code then i showed. can you please tell me why different methods are used to access the main code? is it just preference or is one way actually better coding practice? The second one is used if your importable modules are also scripts that can be executed. That's a bad idea, because it results in the same source file being executed twice, producing distinct classes that break typechecking. If you have myprogram.py containing the following source code, and execute it as a program, it will terminate with an uncaught MyError. class MyError(Exception): pass import myprogram try: raise myprogram.MyError except MyError: pass Why would you do this though? Is the source of the problem not really that you're importing a module inside itself? Why would you generally want to do this? Furthermore the 'if __name__ == __main__' idiom is explicitly part of Python and generally well accepted (so it seems to me at least), so the above seems like a fairly esoteric objection to its use? Thanks, Walter ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 22 October 2012 12:54, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Mon, Oct 22, 2012 at 7:45 AM, Matthew Ngaha chigga...@gmail.com wrote: Hi Devin, your context was missing the crucial part showing the 2nd one: they call the main program by simply calling the main function. I've also seen a more complcated: if __name__ == '__main__': main() the 2nd one usually includes a lot more code then i showed. can you please tell me why different methods are used to access the main code? is it just preference or is one way actually better coding practice? The second one is used if your importable modules are also scripts that can be executed. That's a bad idea, because it results in the same source file being executed twice, producing distinct classes that break typechecking. If you have myprogram.py containing the following source code, and execute it as a program, it will terminate with an uncaught MyError. There is nothing wrong with having a script that can also be imported (or a module that can also be executed). I do this often, particularly when I have a number of related scripts in the same folder as one another. Python allows a .py file to represent a script or a module. Python itself maintains no formal distinction between scripts and modules. Adding if __name__ == __main__ allows your script/module to determine whether it is being executed or imported and do different things in each case. One case where this is good is to add a test suite to a module. For example my module defines some functions that can be imported by other modules. It also defines some code to test that the functions it defines are working properly. I don't want to run the tests every time I import the module but I can run them conveniently if I put them in a __name__ == __main__ block and then execute the module as a script every now and then. Both the unittest and doctest modules supply functions specifically for use in this way. Another common case is that your module defines some functions that are of general use but you also make it possible to run the module as a script in order to perform some of the most common operations that your module performs. There are several examples in the Python standard library that do this for example pdb and cProfile. It is also sometimes useful to define a number of scripts that are in the same directory but share some code by importing one another. You need the if __name__ ==__main__ block for this. The problem that Devin is referring to only happens in certain odd situations where the script that you run ends up importing itself (perhaps indirectly). I have never personally had that problem though. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 9:35 AM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: It is also sometimes useful to define a number of scripts that are in the same directory but share some code by importing one another. You need the if __name__ ==__main__ block for this. The problem that Devin is referring to only happens in certain odd situations where the script that you run ends up importing itself (perhaps indirectly). I have never personally had that problem though. Just to clarify that I'm following you, would you count the following as a script importing itself 'indirectly'? Assume two modules in the same directory, mod1.py and mod2.py, can both act as the main entry point, and both import each other (assume no circular import problem). Then if mod1.py runs as __main__ and imports mod2, and mod2.py imports mod1, there are 2 copies of the classes and functions defined in mod1.py. For example, there's __main__.MyError vs mod1.MyError. I agree there's no problem if no other module imports the module currently running as __main__. In that case, there's nothing technically wrong with mixing script functionality into a module that can also be imported. Self-contained test cases are a clear example of this. Also, modules can import __main__, but it's not used frequently. I count 16 cases of this in the 2.7.3 standard library, for introspective usage such as in idlelib, cProfile, pdb, etc. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 11:14 AM, eryksun eryk...@gmail.com wrote: Just to clarify that I'm following you, would you count the following as a script importing itself 'indirectly'? Assume two modules in the same directory, mod1.py and mod2.py, can both act as the main entry point, and both import each other (assume no circular import problem). Then if mod1.py runs as __main__ and imports mod2, and mod2.py imports mod1, there are 2 copies of the classes and functions defined in mod1.py. For example, there's __main__.MyError vs mod1.MyError. I forgot to specify that __main__ can reference mod1 via mod2.mod1, so my question is if this indirection via mod2 is what you meant by importing itself 'indirectly'. Thanks. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 22 October 2012 16:14, eryksun eryk...@gmail.com wrote: On Mon, Oct 22, 2012 at 9:35 AM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: It is also sometimes useful to define a number of scripts that are in the same directory but share some code by importing one another. You need the if __name__ ==__main__ block for this. The problem that Devin is referring to only happens in certain odd situations where the script that you run ends up importing itself (perhaps indirectly). I have never personally had that problem though. Just to clarify that I'm following you, would you count the following as a script importing itself 'indirectly'? Yes. Assume two modules in the same directory, mod1.py and mod2.py, can both act as the main entry point, and both import each other (assume no circular import problem). They both import each other. That is a circular import and it can create problems. My advice for this issue is not avoid importable scripts but rather avoid circular imports (a good idea anyway). Then if mod1.py runs as __main__ and imports mod2, and mod2.py imports mod1, there are 2 copies of the classes and functions defined in mod1.py. For example, there's __main__.MyError vs mod1.MyError. I agree that this would be problematic. However, if mod1.py and mod2.py both import each another then neither of them is really an independent script. There is no reason why any common code should not be moved into a third module to avoid the circular import. I have used importable scripts many times and never had this problem because I have never used them with circular imports. Typically the action of running the module as a script provides a command line interface that does some common but useful thing with the code in that same module (rather than code imported from other modules). An example from a recent project: I have a module that defines functions for interacting with a particular database. It can be imported by scripts that perform computations based on the contents of the database. It can also be run as a script in which case it provides a command line interface to query/export the contents of the database. This particular module does not import any other modules within the same project so it will never have the circular import problem. There are other importable scripts that depend on the database module but they also don't use circular imports. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 2:26 PM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: They both import each other. That is a circular import and it can create problems. My advice for this issue is not avoid importable scripts but rather avoid circular imports (a good idea anyway). I agree it's better to refactor common code to a separate module, but if you search the net you'll find lots of well-intentioned advice on how to make circular import scenarios work. You had said define a number of scripts that are in the same directory but share some code by [importing one another], and I wanted to be clear about how that related to your comment about the main module indirectly importing itself. Thanks for taking the time to clear that up. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 22/10/12 22:45, Matthew Ngaha wrote: In many of the tutorial examples ive come across, the main code's program is never at the top level, but always in a function of some sort. i understand why but, there is always a different way used to access the main code, i want to know which is the best. main() main's code #top level main() Not that. That unconditionally executes main the first time you access the module, *regardless* of whether it is being run as a script or being imported. That is nearly always the wrong thing to do. It's particularly awful because importing the module for a second time will not re-run the main program. You have to exit Python, restart it, and then re-import the module again. they call the main program by simply calling the main function. I've also seen a more complcated: if __name__ == '__main__': main() This one is best. It only runs the main function when the module is being run as a script. the 2nd one usually includes a lot more code then i showed. Yes, that's because when people write proper scripts, they tend to write code for handling errors, checking the command line arguments, etc. When people just slap together a lazy always execute the main function, they tend to not bother. Or perhaps that's just how I work :) -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 6:15 PM, Steven D'Aprano st...@pearwood.info wrote: Not that. That unconditionally executes main the first time you access the module, *regardless* of whether it is being run as a script or being imported. That is nearly always the wrong thing to do. Recently I've become a fan of executable packages. In __main__.py, it's always the right thing to do. -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 9:23 AM, Walter Prins wpr...@gmail.com wrote: Devin, On 22 October 2012 12:54, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Mon, Oct 22, 2012 at 7:45 AM, Matthew Ngaha chigga...@gmail.com wrote: the 2nd one usually includes a lot more code then i showed. can you please tell me why different methods are used to access the main code? is it just preference or is one way actually better coding practice? The second one is used if your importable modules are also scripts that can be executed. That's a bad idea, because it results in the same source file being executed twice, producing distinct classes that break typechecking. If you have myprogram.py containing the following source code, and execute it as a program, it will terminate with an uncaught MyError. class MyError(Exception): pass import myprogram try: raise myprogram.MyError except MyError: pass Why would you do this though? Is the source of the problem not really that you're importing a module inside itself? Why would you generally want to do this? Furthermore the 'if __name__ == __main__' idiom is explicitly part of Python and generally well accepted (so it seems to me at least), so the above seems like a fairly esoteric objection to its use? It's that your module was imported, while it was executed as a script. It's fairly common for modules to import each other for various reasons. Especially if one module is executable, it might do circular imports for the purposes of its execution (tests or what have you), and these break because the reverse part of the circle actually create a new module rather than doing a circular import. This kind of behavior is pretty much impossible if your scripts are scripts, and your modules are modules. The only downside is that you have to import the code you use in your scripts. -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 22/10/12 22:54, Devin Jeanpierre wrote: On Mon, Oct 22, 2012 at 7:45 AM, Matthew Ngahachigga...@gmail.com wrote: the 2nd one usually includes a lot more code then i showed. can you please tell me why different methods are used to access the main code? is it just preference or is one way actually better coding practice? The second one is used if your importable modules are also scripts that can be executed. That's a bad idea, I disagree. See below. because it results in the same source file being executed twice, Not so. That only happens if your module is executed *and* imported simultaneously. It is perfectly safe to write a module which can be run as a script, or imported, so long as you don't do both at the same time from within a single Python process. Excluding test scripts, I find 145 modules in the Python 2.5 standard library, and 127 in the Python 3.3 std lib, that use the if __name__ == idiom to be both importable and runnable as a script. producing distinct classes that break typechecking. If you have myprogram.py containing the following source code, and execute it as a program, it will terminate with an uncaught MyError. class MyError(Exception): pass import myprogram try: raise myprogram.MyError except MyError: pass Then don't do that. There are very few reasons for importing a module from within itself, and those reasons are relatively advanced, e.g.: - circular imports - some types of introspection If you do that, and the module directly or indirectly imports itself while it is running as a script, you may run into trouble. But writing a dual-purpose module that is usable as an importable module or as a stand-alone script is not problematic in itself. So, in any circumstance where you would use the second one, it's because you're in a situation where bad things are happening. The bad situation is when you have circular imports, including the case where your module imports itself. So use the first one always. This is contrary to the advice given in the Fine Manual: http://docs.python.org/library/__main__.html More from Python's creator, GvR, and the Effbot: http://www.artima.com/forums/flat.jsp?forum=106thread=4829 http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 8:42 PM, Steven D'Aprano st...@pearwood.info wrote: If you do that, and the module directly or indirectly imports itself while it is running as a script, you may run into trouble. But writing a dual-purpose module that is usable as an importable module or as a stand-alone script is not problematic in itself. Yes. However, it is somewhat hard to guarantee that a module won't indirectly import itself for a large codebase. It certainly sometimes happens by accident. So use the first one always. This is contrary to the advice given in the Fine Manual: http://docs.python.org/library/__main__.html More from Python's creator, GvR, and the Effbot: http://www.artima.com/forums/flat.jsp?forum=106thread=4829 http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm It's very easy for me to overstate my case. Maybe I even have already, just because I wasn't taking very long to explain myself. I have concrete reasons to not use this form. It comes up very rarely, and is not a large issue. Heck, it's not even really worth making a big fuss over. But there is the occasional mess-up where it happens. I am also aware that what I prefer goes against idiom. But to me, this minor concrete use-case beats out idiom. After all, what is the benefit to following this particular idiom? I believe the answer is, you get to use one .py file instead of two -- and as I see it, the cases where circular imports might be a problem are precisely those where you don't care too much about the number of files, because you're writing a package anyway. As it happens, I sort of stumbled across this worldview when I was forced into separating modules from scripts with the new __main__ submodule system to make executable packages (So it's even possible for Pythonic code to forgo the name check). It really struck something in me, and I had a (subjective) realization that modules and scripts are fundamentally different, and that there's very little benefit to conflating them. I realize this isn't really a view that's mainstream for Python. Also, I realize it's not a particularly interesting realization -- any C# programmer would tell you this. And presumably, Pythonistas as a group would disagree. (As an aside, I find it interesting how much more beautiful GvR's code in that post becomes if you consider turning the common bits into a @main decorator). -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On 23/10/12 11:25, Devin Jeanpierre wrote: On Mon, Oct 22, 2012 at 6:15 PM, Steven D'Apranost...@pearwood.info wrote: Not that. That unconditionally executes main the first time you access the module, *regardless* of whether it is being run as a script or being imported. That is nearly always the wrong thing to do. Recently I've become a fan of executable packages. In __main__.py, it's always the right thing to do. I would disagree there too. I think that unconditionally running main is the wrong thing to do, except perhaps in the most trivial quick-and- dirty scripts. But if the script it that trivial, you might not even bother with a main function at all. package.__main__.py is designed to be run as a script, and not to be imported. But that doesn't mean that there's no good purpose for importing it. If your package is non-trivial, you ought to have tests for it, including package.__main__. Those tests will probably want to import the module, not necessarily run it as a script. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which is better Practice and why
On Mon, Oct 22, 2012 at 8:54 PM, Steven D'Aprano st...@pearwood.info wrote: Recently I've become a fan of executable packages. In __main__.py, it's always the right thing to do. I would disagree there too. I think that unconditionally running main is the wrong thing to do, except perhaps in the most trivial quick-and- dirty scripts. But if the script it that trivial, you might not even bother with a main function at all. package.__main__.py is designed to be run as a script, and not to be imported. But that doesn't mean that there's no good purpose for importing it. If your package is non-trivial, you ought to have tests for it, including package.__main__. Those tests will probably want to import the module, not necessarily run it as a script. I think you will find that it is never the case that __name__ != '__main__' in a file called __main__.py. If I want to test something, I put it in another file. My __main__.py file will generally look something like this: from . import game game.Game().run() -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
