Re: [Tutor] PyMOTW: difflib

[Kent Johnson]

Alan Gauld wrote:
> "Dick Moores" <[EMAIL PROTECTED]> wrote 
> 
>> Could you give a blow-by-blow on the dis.dis()?
> 
> I'll have a go but I've never actually studied Python P-Code, 
> this is based on my 8080 and 68000 assembler experience!

Some hits, some misses. I'll correct to the best of my knowledge.

One thing you need to know is that the Python VM is a stack machine, not 
  register-based. Think Forth or Postscript, not 8080.
> 
>> In [22]: def f(x):
>>   : if x%2:
>>   : return "odd"
>>   : else:
>>   : return "even"
>>   :
>>   :
>>
>> In [23]: dis.dis(f)
>>  2   0 LOAD_FAST0 (x)
>>  3 LOAD_CONST   1 (2)
> 
> load x and 2 into two temporary storage areas
> (registers in conventional assembler)

No, it pushes x and 2 onto the stack. 0 and 1 are the indices into the 
local variable list (in the stack frame) and the function's constant 
list (part of the function). The values in parentheses are helpful 
comments by dis, not part of the code itself.
> 
>>  6 BINARY_MODULO
> 
> Do the modulo operation and store the result in a temporary 
> location (accumulator in assembler)

The arguments are the top two items on the stack, the result is left on 
the stack.
> 
>>  7 JUMP_IF_FALSE8 (to 18)
> 
> If the result is zero(false)

Yes, but not the way you think it works. It is checking the top of the 
stack (TOS in the docs), not some flag set by the modulo operation the 
way an 8080 would.

> go to instruction 18 
> (I don't know what the 8 signifies, I thought a
> program counter offset but that would make it 
> a jump to 15...)

Probably an offset to the PC value of the next instruction. Again, the 
part in parens is a helpful comment.
> 
>> 10 POP_TOP
> 
> else (ie modulo result is non zero) pop the top - I'm 
> not sure what that does exactly but I assume it pops 
> a value of some stack into the accumulator area?

Just pops the stack, the value is no longer needed after the test.
> 
>>  3  11 LOAD_CONST   2 ('odd')
>> 14 RETURN_VALUE
> 
> Load 'odd' into storage area 2 and then return it?

Load odd *from* offset 2 to the stack and return it.
> 
>> 15 JUMP_FORWARD 5 (to 23)
> 
> Not sure about this bit...

I think this is junk. The Python compiler does very little optimization, 
even simple peephole optimizations. This looks like boilerplate code 
emitted to skip the 'else' clause. The boilerplate is left in even 
though the code is not reachable.
> 
>>>>   18 POP_TOP
> 
> This was target of our previous jump if false instruiction
> 
>>  5  19 LOAD_CONST   3 ('even')
>> 22 RETURN_VALUE
> 
> So it loads 'even' and then returns it to the caller.
> 
>>>>   23 LOAD_CONST   0 (None)
>> 26 RETURN_VALUE
> 
> I assume this is the default return value in case there is no 
> other returns, so not used here.

More leftover junk.

BTW there are several recipes in the cookbook that rely on hacking the 
bytecodes directly from Python. Here are some:
http://tinyurl.com/6xn55p

This one by Raymond Hettinger optimizes references to globals by 
automatically converting them to references to locals. "The speed-up is 
substantial (trading one or more dictionary lookups for a single C-speed 
array lookup)."
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Marc Tompkins]

On Mon, Apr 7, 2008 at 4:52 PM, Alan Gauld <[EMAIL PROTECTED]>
wrote:

> "Dick Moores" <[EMAIL PROTECTED]> wrote
>
> > Could you give a blow-by-blow on the dis.dis()?
>
> I'll have a go but I've never actually studied Python P-Code,
> this is based on my 8080 and 68000 assembler experience!
> ...
> I'm no expert but I'm sure someone else can fix up my errors
> but its very like any other kind of assembler - you just need
> to look up the behaviour of each op code.
>
> Alan G.
>
>
Stop it, you two!  You're bringing back painful memories... I do believe I'm
having a flashback to my Z-80 days... The horror!  The horror!

Has there been any research on assembler-induced PTSD, I wonder?  Thank
Guido for Python, where we can forget all our low-level cares...  or at
least we COULD...

-- 
www.fsrtechnologies.com
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Alan Gauld]

"Dick Moores" <[EMAIL PROTECTED]> wrote 

> Could you give a blow-by-blow on the dis.dis()?

I'll have a go but I've never actually studied Python P-Code, 
this is based on my 8080 and 68000 assembler experience!

> In [22]: def f(x):
>   : if x%2:
>   : return "odd"
>   : else:
>   : return "even"
>   :
>   :
> 
> In [23]: dis.dis(f)
>  2   0 LOAD_FAST0 (x)
>  3 LOAD_CONST   1 (2)

load x and 2 into two temporary storage areas
(registers in conventional assembler)

>  6 BINARY_MODULO

Do the modulo operation and store the result in a temporary 
location (accumulator in assembler)

>  7 JUMP_IF_FALSE8 (to 18)

If the result is zero(false) go to instruction 18 
(I don't know what the 8 signifies, I thought a
program counter offset but that would make it 
a jump to 15...)

> 10 POP_TOP

else (ie modulo result is non zero) pop the top - I'm 
not sure what that does exactly but I assume it pops 
a value of some stack into the accumulator area?

>  3  11 LOAD_CONST   2 ('odd')
> 14 RETURN_VALUE

Load 'odd' into storage area 2 and then return it?

> 15 JUMP_FORWARD 5 (to 23)

Not sure about this bit...

>>>   18 POP_TOP

This was target of our previous jump if false instruiction

>  5  19 LOAD_CONST   3 ('even')
> 22 RETURN_VALUE

So it loads 'even' and then returns it to the caller.

>>>   23 LOAD_CONST   0 (None)
> 26 RETURN_VALUE

I assume this is the default return value in case there is no 
other returns, so not used here.

I'm no expert but I'm sure someone else can fix up my errors 
but its very like any other kind of assembler - you just need 
to look up the behaviour of each op code.

Alan G.

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Dick Moores]

At 02:57 PM 4/7/2008, Dick Moores wrote:
This may be pushing it with you,
but I found the dis.dis() thing fascinating. Here's a bit more complex
function.
 def f(x):
 if x%2:
 return
"odd"
 else:
 return
"even"
Could you give a blow-by-blow on the dis.dis()?
===
In [21]: import dis
   :
Out[21]: 
In [22]: def f(x):
   : if x%2:
   : return
"odd"
   : else:
   : return
"even"
   :
   :
In [23]: dis.dis(f)
  2   0
LOAD_FAST   
0 (x)
 
3
LOAD_CONST  
1 (2)
 
6 BINARY_MODULO
 
7
JUMP_IF_FALSE   
8 (to 18)

10 POP_TOP
  3  11
LOAD_CONST  
2 ('odd')

14 RETURN_VALUE

15
JUMP_FORWARD
5 (to 23)
    >>   18
POP_TOP

  5  19
LOAD_CONST  
3 ('even')

22 RETURN_VALUE
    >>   23
LOAD_CONST  
0 (None)

26 RETURN_VALUE
=
Kent, never mind. I just realized that all those LOAD_FAST's, etc., are
defined in

http://docs.python.org/lib/bytecodes.html .
Of course, should you want to say something, I'm always ready to listen
to you. 
And that goes for Alan as well..
Dick

  

UliPad <>:

http://code.google.com/p/ulipad/


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Dick Moores]

At 12:33 PM 4/7/2008, Kent Johnson wrote:
In [13]: import dis
In [15]: def foo():
    : a = 10
In [16]: dis.dis(foo)
  
2   0
LOAD_CONST  
1 (10)
  
3
STORE_FAST  
0 (a)
  
6
LOAD_CONST  
0 (None)
  
9 RETURN_VALUE
The STORE_FAST instruction is storing the (reference to) the value 10 at

offset 0 in the local storage section of the stack
frame.
I looked up "offset" in Wikipedia:

Offset (computer science)
This article describes the computer science term. 
In computer science, an offset within an array or other data structure
object is an integer indicating the distance (displacement) from the
beginning of the object up until a given element or point, presumably
within the same object. The concept of a distance is valid only if all
elements of the object are the same size (typically given in bytes or
words).
In computer engineering and low-level programming (such as assembly
language), an offset usually denotes the number of address locations
added to a base address in order to get to a specific absolute address.
In this meaning of offset, only the basic address unit, usually the 8-bit
byte, is used to specify the offset's size. In this context an offset is
sometimes called a relative address.
For example, given an array of characters A, containing abcdef, one can
say that the element containing the letter 'c' has an offset of 2 from
the start of A.
Retrieved from
"
http://en.wikipedia.org/wiki/Offset_%28computer_science%29"
===
This may be pushing it with you, but I found the dis.dis() thing
fascinating. Here's a bit more complex function.
 def f(x):
 if x%2:
 return
"odd"
 else:
 return
"even"
Could you give a blow-by-blow on the dis.dis()?
===
In [21]: import dis
   :
Out[21]: 
In [22]: def f(x):
   : if x%2:
   : return
"odd"
   : else:
   : return
"even"
   :
   :
In [23]: dis.dis(f)
  2   0
LOAD_FAST   
0 (x)
 
3
LOAD_CONST  
1 (2)
 
6 BINARY_MODULO
 
7
JUMP_IF_FALSE   
8 (to 18)

10 POP_TOP
  3  11
LOAD_CONST  
2 ('odd')

14 RETURN_VALUE

15
JUMP_FORWARD
5 (to 23)
    >>   18
POP_TOP
  5  19
LOAD_CONST  
3 ('even')

22 RETURN_VALUE
    >>   23
LOAD_CONST  
0 (None)

26 RETURN_VALUE
=
Thanks,
Dick Moores


  

UliPad <>:

http://code.google.com/p/ulipad/


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Kent Johnson]

Alan Gauld wrote:
> "Dick Moores" <[EMAIL PROTECTED]> wrote 

>> Is a name the same as anything defined? 
> 
> Yes pretty much anything that has a name.
> 
> Its worth spending the time reading and playing with 
> this because its a fundamental concept in Python 
> and once understood many of the seeming anomolies 
> become clear.

Yes, I strongly agree. There is a bit of an aha moment when you realize, 
"oh, they are all just names in namespaces."

> I tend to think of it this way. Python does most things 
> using dictionaries. The names we define in our code 
> are merely keys into some semi-hidden dictionaries.
> The values of the dictionaries are the things the names 
> reference - values, classes, functions etc

Nitpick: I think of it that way too but it is not always correct. Many 
names are stored as key/value pairs in dicts but local variables are 
stored in a pre-allocated array and accessed by offset. There are 
probably other exceptions also. For example:

In [13]: import dis
In [15]: def foo():
: a = 10
In [16]: dis.dis(foo)
   2   0 LOAD_CONST   1 (10)
   3 STORE_FAST   0 (a)
   6 LOAD_CONST   0 (None)
   9 RETURN_VALUE

The STORE_FAST instruction is storing the (reference to) the value 10 at 
offset 0 in the local storage section of the stack frame.

> When you import a module (import x) you reference the 
> module dictionary so still need to derefence it (eg. sys.exit)
> When you import from a module (from x import *) you 
> copy the keys into your local dictionary so you don't 
> need to dereference them.
> 
> That's almost certainly not a technically correct explanation 
> of how importing really works but its conceptually how I 
> think about it. :-)

My take on this:

When you import a module, you bind a name in the current namespace to 
the module object. So
   import x
binds the name 'x' to the module object. Attributes of the module - any 
names at global scope in the module - are available using normal 
attribute access syntax, e.g.
   x.a

You can also import module x and bind it to a different name, e.g.
   import x as xx

When you import specific attributes of a module, then the value of the 
module attribute is bound to a name in the current namespace. For example
   from x import a
binds x.a to the name a. Here again, you can use a different name:
   from x import a as b
binds x.a to the name b.

   from x import *
is similar except every public attribute of x is bound to a local name.

HTH someone!
Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Alan Gauld]

"Dick Moores" <[EMAIL PROTECTED]> wrote 

> I'm having trouble with the concept of a name in Python.
> 
> Is a name the same as anything defined? 

Yes pretty much anything that has a name. It could be 
a variable, a class, a module, a function, anything that 
you can refer to or can refer to an object (bearing in mind 
that modules, functions and classes are objects in Python!)

Its worth spending the time reading and playing with 
this because its a fundamental concept in Python 
and once understood many of the seeming anomolies 
become clear.

I tend to think of it this way. Python does most things 
using dictionaries. The names we define in our code 
are merely keys into some semi-hidden dictionaries.
The values of the dictionaries are the things the names 
reference - values, classes, functions etc

When you import a module (import x) you reference the 
module dictionary so still need to derefence it (eg. sys.exit)
When you import from a module (from x import *) you 
copy the keys into your local dictionary so you don't 
need to dereference them.

That's almost certainly not a technically correct explanation 
of how importing really works but its conceptually how I 
think about it. :-)

HTH,

-- 
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Dick Moores]

At 12:23 AM 4/7/2008, Alan Gauld wrote:
>"Dick Moores" <[EMAIL PROTECTED]> wrote in message
>
> > Even if there weren't a path problem, my understanding is that
> > from difflib_data import *   would only import the functions in
> > difflib_data.py; There are none.
>
>import imports names. Any names, not just functions.
>After all, function names are just names like any other
>
>sqr = lambda x: x*x
>
>is exactly the same as
>
>def sqr(x): return x*x
>
>In both cases import will give you access to sqr as a name
>
>So in your case the names text1, text2, text1_lines and
>text2_lines will all be made available within your script.

I'm having trouble with the concept of a name in Python.

Is a name the same as anything defined? Such as

a, where a = 5
b, where b = [3,4,5]
c, where c = "Alan"
five(), where  def five(): return 5
sqr, where sqr = lambda x: x*x
square, where def sqr(x): return x*x

Are there other kinds of possibilities?
names of classes?
names of modules?

Dick



UliPad <>: http://code.google.com/p/ulipad/ 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Alan Gauld]

"Dick Moores" <[EMAIL PROTECTED]> wrote in message 

> Even if there weren't a path problem, my understanding is that   
> from difflib_data import *   would only import the functions in 
> difflib_data.py; There are none.  

import imports names. Any names, not just functions.
After all, function names are just names like any other

sqr = lambda x: x*x

is exactly the same as

def sqr(x): return x*x

In both cases import will give you access to sqr as a name

So in your case the names text1, text2, text1_lines and 
text2_lines will all be made available within your script.

HTH,


-- 
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Dick Moores]

At 04:39 PM 4/6/2008, Alan Gauld wrote:
"Dick Moores"
<[EMAIL PROTECTED]> wrote
>> > Copied the data into my demo_pymotw-difflib.py,
>> > deleted the line "from difflib_data import
*",
>> > and ran it
>>
>>I'm curious. Why did you do that?
> OK, I put it back in, changing the filename to 
> demo_pymotw-difflib_error.py.
> See
<
http://py77.python.pastebin.com/f5c3a37ce> for the code and
the
> ImportError: No module named difflib_data.
You still have the copied data which will overwrite the values
imported however to import it you need to have the path set
up correctly. Is the difflib_data.py in the same folder as
difflib.py?
No. The paths are E:\PythonWork\PyMOTW-1.48\difflib\difflib_data.py
and E:\Python25\Lib\difflib.py
Even if there weren't a path problem, my understanding is
that   from difflib_data import *   would only import
the functions in difflib_data.py; There are none.  See
difflib_data.py at
<
http://py77.python.pastebin.com/f5e2c73a5>. Am I wrong?
Ah ha! I AM wrong, aren't I. I just realized that Hellman's intention was
to run difflib_differ.py
(<
http://py77.python.pastebin.com/f7853ef8>), which IS in the same
folder, difflib, as difflib_data.py.  Works fine.  I didn't
need to make my own script. 
I'm still puzzled, though. How can   from difflib_data import
*   work?  
Let's see. Say I have a script, some_variables.py
===some_variables.py=
#!/usr/bin/env python
a = 1
b = 2
c = 3
=end of some_variables.py
And a script, importer.py
=importer.py=
#!/usr/bin/env python
from some_variables.py import *
print a, b, c, a*b*c
end of importer.py===
And by golly, I do get
> "E:\Python25\pythonw.exe" -u
"E:\PythonWork\importer.py" 
1 2 3 6
Sorry, folks, but I'm truly amazed. How come I didn't know this
before??
Thanks, Alan, you've opened my eyes.
Dick

  

UliPad <>:

http://code.google.com/p/ulipad/


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Alan Gauld]

"Dick Moores" <[EMAIL PROTECTED]> wrote
>> > Copied the data into my demo_pymotw-difflib.py,
>> > deleted the line "from difflib_data import *",
>> > and ran it
>>
>>I'm curious. Why did you do that?

> OK, I put it back in, changing the filename to 
> demo_pymotw-difflib_error.py.
> See  for the code and the
> ImportError: No module named difflib_data.

You still have the copied data which will overwrite the values
imported however to import it you need to have the path set
up correctly. Is the difflib_data.py in the same folder as difflib.py?


Your code should only need to be:

a.. import difflib
a.. from difflib_data import *
a..
a.. d = difflib.Differ()
a.. diff = d.compare(text1_lines, text2_lines)
a.. print '\n'.join(list(diff))

And provided both modules are visible to Python it should work.
Copying the data works too but I was just curious why you
needed to...

Alan G.


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Dick Moores]

At 02:22 PM 4/6/2008, Alan Gauld wrote:

>"Dick Moores" <[EMAIL PROTECTED]> wrote
>
> > I downloaded that compressed file and found difflib_data.py (see it
> > at ). Copied the data
> > into
> > my demo_pymotw-difflib.py, deleted the line "from difflib_data
> > import
> > *", and ran it
>
>I'm curious. Why did you do that?
>Why not just leave the import as it was?

OK, I put it back in, changing the filename to demo_pymotw-difflib_error.py.
See  for the code and the 
ImportError: No module named difflib_data.

Or is there something I don't understand here?

Dick



UliPad <>: http://code.google.com/p/ulipad/ 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Alan Gauld]

"Dick Moores" <[EMAIL PROTECTED]> wrote

> I downloaded that compressed file and found difflib_data.py (see it
> at ). Copied the data 
> into
> my demo_pymotw-difflib.py, deleted the line "from difflib_data 
> import
> *", and ran it

I'm curious. Why did you do that?
Why not just leave the import as it was?

Alan G. 


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Dick Moores]

At 08:24 AM 4/6/2008, Martin Walsh wrote:
>Dick Moores wrote:
> > See < http://blog.doughellmann.com/2007/10/pymotw-difflib.html>
> >
> > And my try with the Differ example,
> > , which also shows the error,
> >
> > "E:\Python25\pythonw.exe" -u "E:\PythonWork\demo_pymotw-difflib.py"
> > Traceback (most recent call last):
> >  File "E:\PythonWork\demo_pymotw-difflib.py", line 12, in 
> >from difflib_data import *
> > ImportError: No module named difflib_data
> >
> > What is difflib_data ?
>
>It is the example data provided with the PyMOTW tutorial.
>
>Near the top of the article (from the link you provided) you'll see the
>heading "Test Data". I assume the author wants you to copy and paste the
>source into a new file named difflib_data.py in your working dir.
>Alternatively, it looks like you can download all the source and example
>data for all PyMOTWs in one compressed file:
>http://www.doughellmann.com/projects/PyMOTW/

Thanks very much for figuring that out for me, and so clearly explaining it.

I downloaded that compressed file and found difflib_data.py (see it 
at ). Copied the data into 
my demo_pymotw-difflib.py, deleted the line "from difflib_data import 
*", and ran it. See it and the output at 


Dick Moores



UliPad <>: http://code.google.com/p/ulipad/ 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Martin Walsh]

Dick Moores wrote:
> See < http://blog.doughellmann.com/2007/10/pymotw-difflib.html>
> 
> And my try with the Differ example, <
> http://py77.python.pastebin.com/f41ec1ae8>, which also shows the error,
> 
> "E:\Python25\pythonw.exe" -u "E:\PythonWork\demo_pymotw-difflib.py"
> Traceback (most recent call last):
>  File "E:\PythonWork\demo_pymotw-difflib.py", line 12, in 
>from difflib_data import *
> ImportError: No module named difflib_data
> 
> What is difflib_data ?

It is the example data provided with the PyMOTW tutorial.

Near the top of the article (from the link you provided) you'll see the
heading "Test Data". I assume the author wants you to copy and paste the
source into a new file named difflib_data.py in your working dir.
Alternatively, it looks like you can download all the source and example
data for all PyMOTWs in one compressed file:
http://www.doughellmann.com/projects/PyMOTW/

PyMOTW has a cheese shop entry also (http://pypi.python.org/), so one
would assume you could get the source with easy_install as well, but
I've never tried it.

HTH,
Marty


> 
> Thanks,
> 
> Dick Moores

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] PyMOTW: difflib

[Dick Moores]

See
<
http://blog.doughellmann.com/2007/10/pymotw-difflib.html>
And my try with the Differ example,
<
http://py77.python.pastebin.com/f41ec1ae8>, which also shows the
error,
"E:\Python25\pythonw.exe" -u
"E:\PythonWork\demo_pymotw-difflib.py" 
Traceback (most recent call last):
 File "E:\PythonWork\demo_pymotw-difflib.py", line 12, in

   from difflib_data import *
ImportError: No module named difflib_data
What is difflib_data ?
Thanks,
Dick Moores




___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor