Re: [Tutor] Understanding (Complex) Modules
(I meant to post this to both the list and Steven back a week or so ago. I missed the list, so am posting it here now. Unfortunately, I'm again in a position where I cannot respond for several days, but I will be back early next week to examine some posts that followed this.) === Thanks. I'm on the road for a few days and will be able to read this more carefully when I get back. I had a feeling from my first attempts of looking at MPL and other heavy-duty libs that it was going to require work to dig out what I needed from the module itself. In my personal view, the usage and learning documents are fairly limiting. Perhaps on in MPL, which is less traditional than numpy and the others, which are founded on common and historic implementations of math and science from older languages. From: Steven D'Aprano st...@pearwood.info To: tutor@python.org Sent: Thu, March 4, 2010 6:24:15 PM Subject: Re: [Tutor] Understanding (Complex) Modules On Thu, 4 Mar 2010 12:24:35 pm Wayne Watson wrote: First a little preamble before my questions. Most of my work in Python has required modifying a program that uses modules that were imported by the original program. I've made some use of modules on a command line like math, and have used the idea of a qualifier. On occasion, I've used examples from matplotlib that required from matplotlib.image import AxesImage. Further, I've created some simple classes, and produced objects with them. No use of inheritance though. So far so good. In a few places, it is said modules are objects. I'm slightly puzzled by that, but in some way it seems reasonable from the standpoint of period notation. So far I have not created a module. Yes you have, you just don't know it. In simple language, a file with Python-usable code in it is a module. There are some technicalities and complexities, but in basic terms, that's all it is. Technically, module refers only to the object which exists inside the Python environment after you call import thing. The import statement does a whole lot of tricks, but in a nutshell it: * finds a file containing code called 'thing' * loads it into memory, executing code as needed * creates a 'module' object to store the code and data in the file * and makes it available to your code Python can create module objects from: compiled Windows DLLs .dll compiled Linux object files .so Python source code .py Python byte code .pyc and .pyo (and .pyw on Windows) Zip files containing any of the above and probably other things as well, but they're the main ones. So any Python file you create (any .py file) is a module, or at least it would be a module if you import it. Ignoring all the various compiled files (.dll, .so, etc) what happens when you run a .py file from the command line (or from IDLE or another IDE). E.g. you type something like: python.exe myscript.py [enter] Python reads the file myscript.py and executes it, but no module object is created. It is possible that a module object *is* created, for internal use, then thrown away when the script is finished. But your script doesn't see the module object. However, if you enter the Python interactive environment, and do this: import myscript # no .py Python loads the file myscript.py into a module object, executing any code in it, and makes it available as a module. Here comes the questions. Recently I began to use matplotlib, scipy, and pylab, mostly matplotlib. I've ground out a few useful pieces of code, but I'm fairly baffled by the imports required to get at various elements, of say, matplotlib (MPL). Some of the MPL examples use of imports make sense, but how the writer pulled in the necessary elements is not. How does one go about understanding the capabilities of such modules? Time, experience, and a lot of hard work. Welcome to programming! If the documentation is good, then read the documentation. If the documentation is lacking, or bad, or even wrong, then read the source code, or search the Internet for a tutorial, or buy a book about it (the numpy people, for example, sell books about numpy). Python makes experimentation easy: there are a lot of powerful introspection facilities in Python. The interactive interpreter is your best friend. You will live with it, eat with it, sleep with it, take it on double-dates, and throw yourself on live grenades to protect it. Whenever I'm programming, I almost always have three or five interactive sessions open for experimentation. The dir() and help() functions are also good friends. In an interactive session: import math dir(math) # prints a list of names in the math module help(math.sin) help(math) Don't feel that you have to understand the entire module before you use it. Many (alas, not all) modules have a reasonably gentle learning curve: you can start using math.sin without needing to know what math.sinh is for. Google and Wikipedia are also your
Re: [Tutor] Understanding (Complex) Modules
Clarifications: A module is a file. It may or may not contain python code. If it does not an exception will be raised when importing. Import executes the module's code exactly the same as if the module had been run as a script (main program). References to module objects are stored in a dict (sys.modules)s. You will notice many modules that have been pre-imported: import sys for x in sys.modules.keys(): print x ... 'copy_reg' 'sre_compile' 'locale' '_sre' 'functools' 'encodings' 'site' '__builtin__' 'operator' '__main__' 'types' 'encodings.encodings' 'abc' 'encodings.cp437' 'errno' 'encodings.codecs' 'sre_constants' 're' '_abcoll' 'ntpath' '_codecs' 'nt' '_warnings' 'genericpath' 'stat' 'zipimport' 'encodings.__builtin__' 'warnings' 'UserDict' 'encodings.cp1252' 'sys' 'codecs' 'os.path' '_functools' '_locale' 'signal' 'linecache' 'encodings.aliases' 'exceptions' 'sre_parse' 'os' So all import sys does is: sys = sys.modules['sys'] Whereas import foo (assuming we refer to foo.py): if 'foo' in sys.modules: foo = sys.modules['foo'] else: compile foo.py if successful: execute the compiled code thus creating a module object if successful: sys.modules['foo'] = the new module object foo = sys.modules['foo'] Herelin lies a gotcha: import foo again does NOT recompile; it just reassigns foo = sys.modules['foo']. reload(foo) will go thru the compile execute assign sequence again. Notice __main__ - that is the module of the main program. sys.modules['__main__'] module '__main__' (built-in) dir(sys.modules['__main__']) ['__builtins__', '__doc__', '__name__', '__package__', 'sys', 'x'] -- Bob Gailer 919-636-4239 Chapel Hill NC ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Understanding (Complex) Modules
On Thu, 4 Mar 2010 12:24:35 pm Wayne Watson wrote: First a little preamble before my questions. Most of my work in Python has required modifying a program that uses modules that were imported by the original program. I've made some use of modules on a command line like math, and have used the idea of a qualifier. On occasion, I've used examples from matplotlib that required from matplotlib.image import AxesImage. Further, I've created some simple classes, and produced objects with them. No use of inheritance though. So far so good. In a few places, it is said modules are objects. I'm slightly puzzled by that, but in some way it seems reasonable from the standpoint of period notation. So far I have not created a module. Yes you have, you just don't know it. In simple language, a file with Python-usable code in it is a module. There are some technicalities and complexities, but in basic terms, that's all it is. Technically, module refers only to the object which exists inside the Python environment after you call import thing. The import statement does a whole lot of tricks, but in a nutshell it: * finds a file containing code called 'thing' * loads it into memory, executing code as needed * creates a 'module' object to store the code and data in the file * and makes it available to your code Python can create module objects from: compiled Windows DLLs .dll compiled Linux object files .so Python source code .py Python byte code .pyc and .pyo (and .pyw on Windows) Zip files containing any of the above and probably other things as well, but they're the main ones. So any Python file you create (any .py file) is a module, or at least it would be a module if you import it. Ignoring all the various compiled files (.dll, .so, etc) what happens when you run a .py file from the command line (or from IDLE or another IDE). E.g. you type something like: python.exe myscript.py [enter] Python reads the file myscript.py and executes it, but no module object is created. It is possible that a module object *is* created, for internal use, then thrown away when the script is finished. But your script doesn't see the module object. However, if you enter the Python interactive environment, and do this: import myscript # no .py Python loads the file myscript.py into a module object, executing any code in it, and makes it available as a module. Here comes the questions. Recently I began to use matplotlib, scipy, and pylab, mostly matplotlib. I've ground out a few useful pieces of code, but I'm fairly baffled by the imports required to get at various elements, of say, matplotlib (MPL). Some of the MPL examples use of imports make sense, but how the writer pulled in the necessary elements is not. How does one go about understanding the capabilities of such modules? Time, experience, and a lot of hard work. Welcome to programming! If the documentation is good, then read the documentation. If the documentation is lacking, or bad, or even wrong, then read the source code, or search the Internet for a tutorial, or buy a book about it (the numpy people, for example, sell books about numpy). Python makes experimentation easy: there are a lot of powerful introspection facilities in Python. The interactive interpreter is your best friend. You will live with it, eat with it, sleep with it, take it on double-dates, and throw yourself on live grenades to protect it. Whenever I'm programming, I almost always have three or five interactive sessions open for experimentation. The dir() and help() functions are also good friends. In an interactive session: import math dir(math) # prints a list of names in the math module help(math.sin) help(math) Don't feel that you have to understand the entire module before you use it. Many (alas, not all) modules have a reasonably gentle learning curve: you can start using math.sin without needing to know what math.sinh is for. Google and Wikipedia are also your friends, although not your *best* friends. (Don't necessarily believe *everything* you read on the Internet.) Don't forget other search engines apart from Google: they're good, but not perfect. MPL seems to have a lot of lower level components. Some of them are laid out over numerous pages as in the form of a, say, English language, description. How does one decipher this stuff. For example, open the module in an editor and start looking at the organization? I thinkthe so called MPL guide ins 900 pages long. Even the numpy guide (reference?), I believe borders on 1000 pages. There must be some way to untangle these complex modules, I would think. Some of the tutorials seem nothing more than a template to follow for a particular problem. So far, looking at the plentiful number of examples of MPL, and probably some of the other modules mentioned above have not provided a lot of insight. Big, complex modules tend to have steep learning curves. There's no
Re: [Tutor] Understanding (Complex) Modules
--- On Wed, 3/3/10, Wayne Watson sierra_mtnv...@sbcglobal.net wrote: From: Wayne Watson sierra_mtnv...@sbcglobal.net Subject: [Tutor] Understanding (Complex) Modules To: tutor@python.org Date: Wednesday, March 3, 2010, 8:24 PM First a little preamble before my questions. Most of my work in Python has required modifying a program that uses modules that were imported by the original program. I've made some use of modules on a command line like math, and have used the idea of a qualifier. On occasion, I've used examples from matplotlib that required from matplotlib.image import AxesImage. Further, I've created some simple classes, and produced objects with them. No use of inheritance though. So far so good. In a few places, it is said modules are objects. I'm slightly puzzled by that, but in some way it seems reasonable from the standpoint of period notation. So far I have not created a module. In Lutz's and Ascher's Learning Python, ed. 2, chap. 16, they describe the following example, among others: module2.py as print starting to load ... import sys def func(): pass class klass: pass print done loading. Their description of its use is quite readable. It shows that there is some more to a module than a list of defs, for example. Here comes the questions. Recently I began to use matplotlib, scipy, and pylab, mostly matplotlib. I've ground out a few useful pieces of code, but I'm fairly baffled by the imports required to get at various elements, of say, matplotlib (MPL). Some of the MPL examples use of imports make sense, but how the writer pulled in the necessary elements is not. How does one go about understanding the capabilities of such modules? MPL seems to have a lot of lower level components. Some of them are laid out over numerous pages as in the form of a, say, English language, description. How does one decipher this stuff. For example, open the module in an editor and start looking at the organization? I thinkthe so called MPL guide ins 900 pages long. Even the numpy guide (reference?), I believe borders on 1000 pages. There must be some way to untangle these complex modules, I would think. Some of the tutorials seem nothing more than a template to follow for a particular problem. So far, looking at the plentiful number of examples of MPL, and probably some of the other modules mentioned above have not provided a lot of insight. Is there some relationship between modules and objects that I'm not seeing that could be of value? -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Stop the illegal killing of dolphins and porpoises. http://www.takepart.com/thecove Wrest the control of the world's fisheries from Japan. Web Page:www.speckledwithstars.net/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor How about importing your foot into your ass, for trying to ask an educated question and answering it through a false addresss by yourself. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
