Re: Why new Python 2.5 feature class C() return old-style class ?

[Bengt Richter]

On Sun, 23 Apr 2006 22:12:01 GMT, [EMAIL PROTECTED] (Bengt Richter) wrote:
[...]
(IMO the proper way to indicate the you don't have a tuple is to use None or 
some other sentinel,
not abuse a perfectly legal tuple value).

  dis.dis(compile('class X:pass','','exec'))
   1   0 LOAD_CONST   0 ('X')   --+-- ought to be 
 LOAD_CONST 0 (None)
Oops, leave the 'X' of course, just replace the next line
   3 BUILD_TUPLE  0 --'
   6 LOAD_CONST   1 (code object X at 02EE7EA0, file 
 , line 1)
   9 MAKE_FUNCTION0
  12 CALL_FUNCTION0
  15 BUILD_CLASS
  16 STORE_NAME   0 (X)
  19 LOAD_CONST   2 (None)
  22 RETURN_VALUE

vs code for class x(something):pass

  dis.dis(compile('class X(object):pass','','exec'))
   1   0 LOAD_CONST   0 ('X')
   3 LOAD_NAME0 (object)
   6 BUILD_TUPLE  1
   9 LOAD_CONST   1 (code object X at 02EFB9A0, file 
 , line 1)
  12 MAKE_FUNCTION0
  15 CALL_FUNCTION0
  18 BUILD_CLASS
  19 STORE_NAME   1 (X)
  22 LOAD_CONST   2 (None)
  25 RETURN_VALUE


Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why new Python 2.5 feature class C() return old-style class ?

[Bengt Richter]

 their features.
so for now put

__metaclass__ = type

once at the top of your module source, and all your

class X:
...

will be interpreted as

class X:
__metaclass__ = type
...

instead of implicitly as

class X:
__metaclass__ = types.ClassType
...


Please repeat this 101 times each morning:
thou shall not use old-style classes for they are deprecated.

(snip)

 So this new syntax is a good way to boost their uses without bother
 with compatibility of existing code IMHO.

It's mostly a good way to add inconsistency and confusion to a situation
that's already confusing enough for newbies.
I don't agree with your idea of inconsistency.

IMO it would be better to explain that a legal basetuple value (empty tuple) is 
currently
being abused as a logical flag to call types.ClassType(clsname, basestuple, 
clsdict)
instead of type(clsname, basestuple, clsdict), and explain that it will be 
corrected, so that
class X():pass will now call the latter, consistent with class X(bases):pass.

Bottom line: IMO class C():pass should create a new-style class,
and the parens serve well as a reminder of which kind it is, whether empty or 
not,
until py3k.

I.e., make it easy for newbies: parens means new-style, no parens means 
old-style, until py3k.

Pontificating pushes my counter-pontificating button; that's the only 
explanation
I have for doing this. I was going to stop wasting time, but find myself
unable as yet fully to abandon scanning clp and python-dev ;-/

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Too Many if Statements?

[Bengt Richter]

On Sat, 11 Feb 2006 15:40:49 -0500, Terry Reedy [EMAIL PROTECTED] wrote:


Steve Holden [EMAIL PROTECTED] wrote in message 
news:[EMAIL PROTECTED]
 Clearly it would be a good idea to remove whatever problem is causing
 the error,

The problem (see my post of the com_backpatch code) is writing a compound 
statement (here a for loop) with a body so large as to require a jump of 
more than 64K bytes in the compiled bytecode (ie, from the test at the top 
of the loop to the code that follows after the loop).  Until the jump limit 
is raised (likely a long wait ;-), the OP must factor some of the code out 
of the loop.

Easy example:

  def test(n):
 ... while True:
 ... try: co = compile('if x:\n'+ n*' a=1\n','','exec')
 ... except Exception,e: break
 ... n += 1
 ... print 'Stopped at n=%s due to %s: %s'%(n, e.__class__.__name__,e)
 ...

get an idea of where to start that:

  import dis
  n=3
  dis.dis(  compile('if x:\n'+ n*' a=1\n','','exec'))
   1   0 LOAD_NAME0 (x)
   3 JUMP_IF_FALSE   22 (to 28)
   6 POP_TOP

   2   7 LOAD_CONST   0 (1)
  10 STORE_NAME   1 (a)

   3  13 LOAD_CONST   0 (1)
  16 STORE_NAME   1 (a)

   4  19 LOAD_CONST   0 (1)
  22 STORE_NAME   1 (a)
  25 JUMP_FORWARD 1 (to 29)
28 POP_TOP
29 LOAD_CONST   1 (None)
  32 RETURN_VALUE
  (2**16-7)/(13-7)
 10921

back off 1

  test(10920)
 Stopped at n=10922 due to SystemError: com_backpatch: offset too large

Decided to test the exact 65536 jump with code chunks of 16 byte-codes and one
chunk at the end to make 16 with the last JUMP_FORWARD.

  n=4095
  dis.dis(  compile('if x:\n'+ n*' a=1+2,4  \n'+' x=0,x','','exec'))
 Traceback (most recent call last):
   File stdin, line 1, in ?
 SystemError: com_backpatch: offset too large
  n=4094
  dis.dis(  compile('if x:\n'+ n*' a=1+2,4  \n'+' x=0,x','','exec'))
   1   0 LOAD_NAME0 (x)
   3 JUMP_IF_FALSE65520 (to 65526)
   6 POP_TOP

   2   7 LOAD_CONST   0 (1)
  10 LOAD_CONST   1 (2)
  13 BINARY_ADD
  14 LOAD_CONST   2 (4)
  17 BUILD_TUPLE  2
  20 STORE_NAME   1 (a)

   3  23 LOAD_CONST   0 (1)

So the corner case of 2**16 is ok. Believe it or not, I once discovered a 
compiler
error based on optimizing a 2**16-involving loop condition as if it were 0 and 
the
loop didn't execute! IIRC, I bumped into it processing an array of records with 
a stride of
exactly 2**n and it thought it could calculate a 16-bit number of strides for 
end of loop.
No good for arraysize/stridesize==2**16 ;-)

If the OP really HAD to, he could always (untested) break

if cond:
too
large
else:
two
large,also
into
if cond:
too
else:
two
if cond:
large
else:
large,also

but that reads gawdawfully. (So, I imagine, does about any code hitting the 
offset limit ;-)

If it's a matter of too many elifs, the OP could break that more readably, e.g. 
(untested)

done=True
if cond:
   bla
elif c2:
   bla 2
...
elif c456:
   bla456
else:
done=False

more, done = not done,True
if more and c457:
bla c457
elif c458:
bla458
...
elif c1012:
bla1012
else:
done = False

more, done = not done,True
... etc

But the OP should find another approach I think,
because this is still disgusting code ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ordered sets operations on lists..

[Bengt Richter]

On Sat, 11 Feb 2006 10:24:04 -0800, [EMAIL PROTECTED] (Alex Martelli) wrote:

Raymond Hettinger [EMAIL PROTECTED] wrote:
   ...
 The intersection step is unnecessary, so the answer can be simplified a
 bit:
 
  filter(set(l2).__contains__, l1)
 [5, 3]
  filter(set(l1).__contains__, l2)
 [3, 5]

...and if one has time to waste, setification being only an
optimization, it can also be removed: filter(l2.__contains__, l1) etc
(very slow for long lists, of course).

Personally, I'd always use (depending on guesses regarding lengths of
lists) [x for x in l1 if x in l2] or the setified equivalent, of course.

Perhaps newbies should be advised that

[x for x in l1 if x in set(l2)]

is not a (well) setified equivalent? I could see them being tempted.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: * 'struct-like' list *

[Bengt Richter]

On Tue, 07 Feb 2006 18:10:05 GMT, [EMAIL PROTECTED] (Bengt Richter) wrote:
[...]
 ernesto.py -
[...]
Just noticed:
substrings = line.split()
if substrings and isinstance(substrings, list) and substrings[0] == 
 'Name:':
   --not needed

str.split always returns a list, even if it's length 1, so that was harmless 
but should be

 if substrings and substrings[0] == 'Name:':

(the first term is needed because ''.split() = [], to avoid [][0])
Sorry.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: module with __call__ defined is not callable?

[Bengt Richter]

On Thu, 9 Feb 2006 10:07:49 +1100, Delaney, Timothy (Tim) [EMAIL PROTECTED] 
wrote:

Steven D'Aprano wrote:

 That's not a _reason_, it is just a (re-)statement of fact. We know
 that defining a __call__ method on a module doesn't make it callable.
 Why not? The answer isn't because defining a __call__ method on a
 module or an instance doesn't make it callable, that's just avoiding
 the question.=20

My reading of the OP sounded like he wanted to know the *technical*
reason for why it doesn't work - which is what I provided.

If you can come up with a convincing argument, after re-reading the OP
(as I've just done), that that is *not* what he meant, I'm all ears.

While you can't put __call__ on the builtin module type,
you _can_ put it on a subclass:

  class M(type(__builtins__)):
 ...def __call__(self): return '%r was called'%self
 ...
  m = M('M')
  m()
 module 'M' (built-in) was called

Not that I can see why the OP would want to spell any module access module().

OTOH, I could see wanting to define properties, and access module.latest_foo
and have it choose dynamically from versioned stuff, or something like that.
Of course you can write module.get_latest_foo() as an ordinary function, and
not be bothered with subclassing and jimmying sys.modules and such.

Properties and methods are both forms of descriptors, so they need to be 
attributes
of the type to work. With a subclass you could do that too.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting line endings

[Bengt Richter]

On 6 Feb 2006 06:35:14 -0800, Fuzzyman [EMAIL PROTECTED] wrote:

Hello all,

I'm trying to detect line endings used in text files. I *might* be
decoding the files into unicode first (which may be encoded using
multi-byte encodings) - which is why I'm not letting Python handle the
line endings.

Is the following safe and sane :

text = open('test.txt', 'rb').read()
if encoding:
text = text.decode(encoding)
ending = '\n' # default
if '\r\n' in text:
text = text.replace('\r\n', '\n')
ending = '\r\n'
elif '\n' in text:
ending = '\n'
elif '\r' in text:
text = text.replace('\r', '\n')
ending = '\r'


My worry is that if '\n' *doesn't* signify a line break on the Mac,
then it may exist in the body of the text - and trigger ``ending =
'\n'`` prematurely ?

Are you guaranteed that text bodies don't contain escape or quoting
mechanisms for binary data where it would be a mistake to convert
or delete an '\r' ? (E.g., I think XML CDATA might be an example).

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Literal Escaped Octets

[Bengt Richter]

On Mon, 06 Feb 2006 04:40:31 GMT, Chason Hayes [EMAIL PROTECTED] wrote:

I am trying to convert raw binary data to data with escaped octets in
order to store it in a bytea field on postgresql server. I could do this
easily in c/c++ but I need to do it in python. I am not sure how to read
and evaluate the binary value of a byte in a long string when it is a non
printable ascii value in python. I read some ways to use unpack from the
struct module, but i really couldn't understand where that would help. I
looked at the MIMIEncode module but I don't know how to convert the object
to a string. Is there a module that will convert the data? It seems to me
that this question must have been answered a million times before but I
can't find anything.

Have you considered just encoding the data as text in hex or base64, e.g.,

  import binascii
  s = '\x00\x01\x02\x03ABCD0123'
  binascii.hexlify(s)
 '000102034142434430313233'
  binascii.b2a_base64(s)
 'AAECA0FCQ0QwMTIz\n'

which is also reversible later of course:
  h = binascii.hexlify(s)
  binascii.unhexlify(h)
 '\x00\x01\x02\x03ABCD0123'
  b64 = binascii.b2a_base64(s)
  binascii.a2b_base64(b64)
 '\x00\x01\x02\x03ABCD0123'

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: * 'struct-like' list *

[Bengt Richter]

On 6 Feb 2006 09:03:09 -0800, Ernesto [EMAIL PROTECTED] wrote:

I'm still fairly new to python, so I need some guidance here...

I have a text file with lots of data.  I only need some of the data.  I
want to put the useful data into an [array of] struct-like
mechanism(s).  The text file looks something like this:

[BUNCH OF NOT-USEFUL DATA]

Name:  David
Age: 108   Birthday: 061095   SocialSecurity: 476892771999

[MORE USELESS DATA]

Name

Does the useful data always come in fixed-format pairs of lines as in your 
example?
If so, you could just iterate through the lines of your text file as in example 
at end [1]


I would like to have an array of structs.  Each struct has

struct Person{
string Name;
int Age;
int Birhtday;
int SS;
}
You don't normally want to do real structs in python. You probably want to 
define
a class to contain the data, e.g., class Person in example at end [1]


I want to go through the file, filling up my list of structs.

My problems are:

1.  How to search for the keywords Name:, Age:, etc. in the file...
2.  How to implement some organized list of lists for the data
structure.

It may be very easy, if the format is fixed and space-separated and line-paired
as in your example data, but you will have to tell us more if not.

[1] exmaple:

 ernesto.py -
class Person(object):
def __init__(self, name):
self.name = name
def __repr__(self): return 'Person(%r)'%self.name

def extract_info(lineseq):
lineiter = iter(lineseq) # normalize access to lines
personlist = []
for line in lineiter:
substrings = line.split()
if substrings and isinstance(substrings, list) and substrings[0] == 
'Name:':
try:
name = ' '.join(substrings[1:]) # allow for names with spaces
line = lineiter.next()
age_hdr, age, bd_hdr, bd, ss_hdr, ss = line.split()
assert age_hdr=='Age:' and bd_hdr=='Birthday:' and 
ss_hdr=='SocialSecurity:', \
'Bad second line after Name: %s line:\n%r'%(name, 
line)
person = Person(name)
person.age = int(age); person.bd = int(bd); person.ss=int(ss)
personlist.append(person) 
except Exception,e:
print '%s: %s'%(e.__class__.__name__, e)
return personlist

def test():
lines = \
[BUNCH OF NOT-USEFUL DATA]

Name:  David
Age: 108   Birthday: 061095   SocialSecurity: 476892771999

[MORE USELESS DATA]

Name: Ernesto
Age: 25 Birthday: 040181  SocialSecurity: 123456789

Name: Ernesto
Age: 44 Brithdy: 040106 SocialSecurity: 123456789

Name

persondata = extract_info(lines.splitlines())
print persondata
ssdict = {}
for person in persondata:
if person.ss in ssdict:
print 'Rejecting %r with duplicate ss %s'%(person, person.ss)
else:
ssdict[person.ss] = person
print 'ssdict keys: %s'%ssdict.keys()
for ss, pers in sorted(ssdict.items(), key=lambda item:item[1].name): 
#sorted by name
print 'Name: %s Age: %s SS: %s' % (pers.name, pers.age, pers.ss)

if __name__ == '__main__': test()
---

this produces output:

[10:07] C:\pywk\clppy24 ernesto.py
AssertionError: Bad second line after Name: Ernesto line:
'Age: 44 Brithdy: 040106 SocialSecurity: 123456789'
[Person('David'), Person('Ernesto')]
ssdict keys: [123456789, 476892771999L]
Name: David Age: 108 SS: 476892771999
Name: Ernesto Age: 25 SS: 123456789

if you want to try this on a file, (we'll use the source itself here
since it includes valid example data lines), do something like:

  import ernesto
  info = ernesto.extract_info(open('ernesto.py'))
 AssertionError: Bad second line after Name: Ernesto line:
 'Age: 44 Brithdy: 040106 SocialSecurity: 123456789\n'
  info
 [Person('David'), Person('Ernesto')]

tweak to taste ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Generators vs. Functions?

[Bengt Richter]

On Sun, 05 Feb 2006 19:14:29 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote:

On Sun, 05 Feb 2006 03:31:24 +, Neil Schemenauer wrote:

 Peter Hansen [EMAIL PROTECTED] wrote:
 More precisely, the state of the function is *saved* when a yield 
 occurs, so you certainly don't *recreate* it from scratch, but merely 
 restore the state, and this should definitely be faster than creating it 
 from scratch in the first place.
 
 Right.  Resuming a generator is faster than calling a function.

Have you actually measured this, or are you just making a wild guess?

According to a short test performed by Magnus Lycka, resuming a generator
takes more time than calling a function. My own test agrees.

Here is my test, using Python 2.3. I've tried to make the test as fair as
possible, with the same number of name lookups in both pieces of test code.

# straight function, two name lookups

 import timeit

 t1 = timeit.Timer(stmt=func.next(), setup=
... class K:
...   pass
...
... def next():
...   return 1
...
... func = K()
... func.next = next
... )

 t1.timeit()
0.63980388641357422


# generator, two name lookups

 t2 = timeit.Timer(stmt=gen.next(), setup=
... def g():
...   while 1: yield 1
...
... gen = g()
... )

 t2.timeit()
0.82081794738769531


# straight function, one name lookup

 t3 = timeit.Timer(stmt=f(), setup=
... def f():
...   return 1
... )
 
 t3.timeit()
0.47273492813110352


# generator, one name lookup 

 t4 = timeit.Timer(stmt=gnext(), setup=
... def g():
...   while 1: yield 1
...
... gnext = g().next
... )

 t4.timeit()
0.55085492134094238


So on the basis of my tests, there is a small, but significant speed
advantage to _calling_ a function versus _resuming_ a generator.

Of course the other advantages of generators often far outweigh the tiny
setup cost each time you call one. In addition, for any complex function
with significant execution time, the call/resume time may be an
insignificant fraction of the total execution time. There is little or no
point in avoiding generators due to a misplaced and foolish attempt to
optimise your code.

I show an advantage favoring generator resumption vs function call:

  from time import clock
  def f(): return clock()
 ...
  def g(): yield clock(); yield clock()
 ...
  max(f()-f() for x in xrange(1))
 -9.2190462142316409e-006
  max(f()-f() for x in xrange(1))
 -9.2190462139818408e-006
  max(float.__sub__(*g()) for x in xrange(1))
 -7.5428559682677587e-006
  max(float.__sub__(*g()) for x in xrange(1))
 -7.5428559682677587e-006
  max(float.__sub__(*g()) for x in xrange(1))
 -7.5428559682677587e-006

(It'll probably go ten times faster on a recent box ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: webbrowser module + urls ending in .py = a security hole?

[Bengt Richter]

On Mon, 30 Jan 2006 16:00:25 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

Blair P. Houghton wrote:
 I'm going to try it out on a remote server later today.

Don't bother.  I've confirmed the behaviour you saw, and that it is not 
what I'd expect either.  My Firefox certainly isn't configured to run 
.py scripts even when invoked with the file: protocol, so webbrowser 
is almost certainly Doing Bad Things on Windows.

The relevant code from webbrowser.py shows this, confirming FuzzyMan's 
suspicions:

class WindowsDefault:
 def open(self, url, new=0, autoraise=1):
 os.startfile(url)

 def open_new(self, url):
 self.open(url)

 I may also try to poke around in webbrowser.py, if possible, to see if
 I can see whether it's selecting the executable for the given
 extension, or passing it off to the OS.  I would think, since Python is
 not /supposed/ to have client-side scripting powers, that even when the
 script is on the client this is bad behavior.

I'd agree.  I suspect this ought to be reported as a security flaw, 
though it would be nice to know what the fix should be before doing so. 
  Anyone know a more suitable approach on Windows than just passing 
things off to startfile()?

 Just don't have the bandwidth, just now.
 
 Anyone got a good regex that will always detect an extension that might
 be considered a script? Or reject all but known non-scripted
 extensions? 

Would it be sufficient in your case merely to allow only .html files to 
be loaded?  Or URLs without .extensions?  Or even just permit only the 
http: protocol?

How about finding the browser via .html association and then letting that
handle the url? E.g., slong the lines of

  import os
  ft = os.popen('assoc .html').read().split('=',1)[1].strip()
  ft
 'MozillaHTML'
  os.popen('ftype %s'%ft).read().split('=',1)[1].strip()
 'D:\\MOZ\\MOZILL~1\\MOZILL~1.EXE -url %1'


Regards,
Bengt Richter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Intro to Pyparsing Article at ONLamp

[Bengt Richter]

On Mon, 30 Jan 2006 16:39:51 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

Christopher Subich wrote:
 Using English, because that's the only language I'm fluent in, consider 
 the sentence:
 
 The horse raced past the barn fell.
 
 It's just one of many garden path sentences, where something that 
 occurs late in the sentence needs to trigger a reparse of the entire 
 sentence.  

I can't parse that at all.  Are you sure it's correct?  Aren't raced 
and fell both trying to be verbs on the same subject?  English surely 
doesn't allow that forbids that sort of thing. (wink)

The computer at CMU is pretty good at parsing. You can try it at
http://www.link.cs.cmu.edu/link/submit-sentence-4.html

Here's what it did with The horse raced past the barn fell. :

Time  0.00 seconds (81.38 total)
Found 2 linkages (2 with no P.P. violations)
  Linkage 1, cost vector = (UNUSED=0 DIS=0 AND=0 LEN=13)

+Xp+
|+Ss---+   |
+-Wd-+   +Js+  |   |
|  +--Ds-+---Mv--+--MVp--++--Ds-+  |   |
|  | |   |   || |  |   |
LEFT-WALL the horse.n raced.v past.p the barn.n fell.v . 

Constituent tree:

(S (NP (NP The horse)
   (VP raced
   (PP past
   (NP the barn
   (VP fell)
   .)

IIUC, that's the way I parse it too ;-)

(I.e., The horse [being] raced past the barn fell.)

BTW, the online response has some clickable elements in the diagram
to get to definitions of the terms.

Regards,
Bengt Richter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: webbrowser module + urls ending in .py = a security hole?

[Bengt Richter]

On 30 Jan 2006 14:39:29 -0800, Paul Boddie [EMAIL PROTECTED] wrote:

Peter Hansen wrote:

 I'd agree.  I suspect this ought to be reported as a security flaw,
 though it would be nice to know what the fix should be before doing so.
   Anyone know a more suitable approach on Windows than just passing
 things off to startfile()?

I wouldn't mind knowing if os.startfile is the best way to open
resources on Windows, and whether there's a meaningful distinction
between opening and editing resources that is exposed through an
existing Python library. My interest is in making the desktop module a
useful successor to webbrowser:

http://www.python.org/pypi/desktop

Of course, since desktop.open leaves the exact meaning of to open to
the user's desktop configuration, if that configuration then causes a
Python program to be executed without some kind of confirmation,
there's a fairly good argument for claiming that the configuration is
broken - yes, it's the classic Microsoft convenience vs. security
dilemma, circa 1998.

For webbrowser, the opportunity to move blame to the user's environment
is somewhat reduced, since the expectation of browsing a Python
program would often be to show the text of that program. Given that
webbrowser, in order to do its work, may rely on some environment
mechanism that doesn't have the same view of browsing programs, there
is a good argument for decoupling the module from those mechanisms
entirely, although I can imagine that the resulting code would struggle
even then to do the right thing.

I suppose a desktop config file with a sequence of regex patterns and 
associated defined actions
could dispatch urls to shell, browser, or custom app as desired, overriding
registry and/or browser settings by being first to decide. E.g., config might
have CSV-style command,params,... lines like

define,editor,C:\WINNT\system32\vimr.cmd %1
define,browser,D:\MOZ\MOZILL~1\MOZILL~1.EXE -url %1
define,savedialog,C:\util\savedialog.cmd %1
urlfilter,r'(?i)(\.py$|\.pyw|.\txt)$',editor
urlfilter,r'(?i)(\.htm[l]?|\.jpg|\.gif|\.png|\.pdf)$',browser
urlfilter.r'(?i).*',savedialog

(I think this is more generally powerful than typical .INI file structure,
since you can define a very simple interpreter to do about anything with the
CSV data rows in order, including nesting things, if you make commands
that enter and exit nests. E.g.,
pushdir,c:\tmp\foo
...
popdir
log,file,c:\temp\foo\log.txt
log,on
...
log,off

etc. etc)
Of course, you can jigger an INI file to contain any info you want also,
even using the windows 
{Get,Write}PrivateProfile{String,Int,Section,SectionNames}
API functions, which like many MS APIs IME of yore seem to work simply if you 
conform to
their usage preconceptions, but punish you with info discovery hell otherwise 
;-)

Regards,
Bengt Richter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: writing large files quickly

[Bengt Richter]

On Fri, 27 Jan 2006 12:30:49 -0800, Donn Cave [EMAIL PROTECTED] wrote:

In article [EMAIL PROTECTED],
 rbt [EMAIL PROTECTED] wrote:
 Won't work!? It's absolutely fabulous! I just need something big, quick 
 and zeros work great.
 
 How the heck does that make a 400 MB file that fast? It literally takes 
 a second or two while every other solution takes at least 2 - 5 minutes. 
 Awesome... thanks for the tip!!!

Because it isn't really writing the zeros.   You can make these
files all day long and not run out of disk space, because this
kind of file doesn't take very many blocks.   The blocks that
were never written are virtual blocks, inasmuch as read() at
that location will cause the filesystem to return a block of NULs.

I wonder if it will also write virtual blocks when it gets real
zero blocks to write from a user, or even with file system copy utils?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using non-ascii symbols

[Bengt Richter]

)
 TIMES(2, 100)
 PLUS(200, 7)
 TIMES(3, 10)
 PLUS(30, 5)
 TIMES(3, 10)
 PLUS(30, 7)
 TIMES(3, 100)
 PLUS(300, 5)
 TIMES(3, 100)
 PLUS(300, 7)
 [25, 27, 205, 207, 35, 37, 305, 307] 

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: generating method names 'dynamically'

[Bengt Richter]

On Fri, 27 Jan 2006 01:20:52 +0100, Daniel Nogradi [EMAIL PROTECTED] wrote:

  Is it possible to have method names of a class generated somehow
 dynamically?
 
  More precisely, at the time of writing a program that contains a class
  definition I don't know what the names of its callable methods should
  be. I have entries stored in a database that are changing from time to
  time and I always want to be able to call (from another program) those
  method names which are at the moment present in the database.

 Sounds somehow more like all you need is to learn about __getattr__ and
 maybe __call__ instead of actually generating methods.  In other words,
 don't generate anything, just intercept attempts to call things that
 were produced by accessing the attributes of an object.

 Whether that would work or not depends on things you haven't said.  The
 above spec is a little unclear, given among other things that you don't
 call method names, that the code in the methods you would presumably
 want to call has to exist somewhere and you haven't described what that
 code would be, and I can't tell what those database entries are really
 all about other than that they somehow refer to the names of things that
 you think you want to call as methods. :-)

 Perhaps an example is in order...


Thanks for all the replies, it became clear that I need to look into
getattr, __getattr__ and __call__. I'll do that, but since you asked,
here is the thing I would like to do in a little more detail:

My database has 1 table with 2 fields, one called 'name' and the other
one called 'age', let's suppose it has the following content, but this
content keeps changing:
While the program is running? Or between runs of the program?
Either you will need to query the database for each pseudo-method call,
or you will be able to prebuild all the methods at startup or anything between.


Alice 25
Bob  24

--- program1.py 

class klass:

 # do the stuff with getattr using the database
 # but in a way that after the database changes
 # I don't need to rewrite this part


inst =3D klass()

-- program2.py --

import program1

# This should print 'Hello, my name is Bob and I'm 24'
program1.inst.Bob()

# This should print 'Hi, I'm 25 and I'm Alice'
program1.inst.Alice()
Those two messages differ by more than (name, age) content.
Where is the different text coming from, if you are building
these responses totally dynamically?


# This should print an error message, since there is no
# field in the database with name=3DJohn
program1.inst.John()

I suppose you want the klass to have more methods than
just the dynamically built/simulated ones? Otherwise, why
not just define a function that takes a name and goes to the
data base to get the data and then prints the message. E.g.,
program1.print_the_message('John')
instead of
program1.inst.John()
No class needed.
OTOH, if you are going to use a class, you might want to name it
capitalized (more conventional) and derive from object, or subclass
from something else if it makes sense. I.e. class Klass(object): ...

What kind of database are you accessing? An RDBMS? A CSV text file?
A directory full of named single-line files (ugh), or?

How often will you access the data? Are you the only one?
Requirements, requirements ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Loading a Python collection from an text-file

[Bengt Richter]

On Mon, 23 Jan 2006 21:00:55 +0200, Ilias Lazaridis [EMAIL PROTECTED] wrote:

within a python script, I like to create a collection which I fill with 
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need 
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
   user.name
   user.age

.

-- 
http://lazaridis.com

I'd use a CSV text file, maybe something like (only tested as far as you see!):

 for_ilias_lazaridis.py --
import csv, types

class Fields(object):
def __init__(self, kvpairs): self.__dict__.update(kvpairs)

class Users(object):
def __init__(self):
self.userlist=[]
def load(self, lineiter):
if isinstance(lineiter, basestring):
lineiter = open(lineiter) # assume it's a file path
csvit = csv.reader(lineiter)
self.colnames = colnames = csvit.next()
typenames = csvit.next()
self.coltypes =coltypes = [getattr(types, name.capitalize()+'Type')
for name in typenames]
for row in csvit:
self.userlist.append(Fields(zip(colnames, (t(s) for t,s in 
zip(coltypes, row)
def __iter__(self): return iter(self.userlist)

def test():
import StringIO
f = StringIO.StringIO(\
name,age
String,Int
peter,16
anton,21
)
users = Users()
users.load(f)
for user in users:
print user.name, user.age
for user in users:
for name in users.colnames:
print '%s=%s,'%(name, getattr(user, name)),
print

if __name__ == '__main__': test()
---

Output:

[ 4:47] C:\pywk\clppy24 for_ilias_lazaridis.py
peter 16
anton 21
name=peter, age=16,
name=anton, age=21,

(the first for user in users loop presumes knowledge of the field names name 
and age.
The second gets them automatically from the names loaded in the load method from
the first line of the text file. The second line expects type names as you see
in the types module, except without the Type suffix.

Perhaps you can adapt for your purposes.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python String Substitution

[Bengt Richter]

On 26 Jan 2006 15:40:47 -0800, Murali [EMAIL PROTECTED] wrote:

In Python, dictionaries can have any hashable value as a string. In
particular I can say

d = {}
d[(1,2)] = Right
d[(1,2)] = Wrong
d[key] = test

In order to print test using % substitution I can say

print %(key)s % d

Is there a way to print Right using % substitution?

print %((1,2))s % d

gives me Wrong. Is there any syntax which will allow me to get
Right using % substitution?

You can modify the dict to try to convert the string to what it is
a source for, by eval, and try that as a key also, if you have no security 
worries
about malevolent format strings:

  class D(dict):
 ... def __getitem__(self, key):
 ... print repr(key)
 ... if key in self: return dict.__getitem__(self, key)
 ... else: return self[eval(key)]
 ...
  d = D()
  d[(1,2)] = Right
  d[key] = test
  print %(key)s % d
 'key'
 test
  print %((1,2))s % d
 '(1,2)'
 (1, 2)
 Right
  d[123] = 'onetwothree'
  print %(123)s % d
 '123'
 123
 onetwothree

Note recursive printing of converted key when the first one fails.
Of course if you _also_ define the string key for Wrong, that will
succeed, so it won't get converted to get Right:

  d[(1,2)] = Wrong
  print %((1,2))s % d
 '(1,2)'
 Wrong
  d[(1,2)]
 (1, 2)
 'Right'

Do you have a real use case? Just curious.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python code written in 1998, how to improve/change it?

[Bengt Richter]

On Wed, 25 Jan 2006 15:50:27 -0600, [EMAIL PROTECTED] wrote:

 
 If they need to resume their calculations from where they left off
 after the last yield.

Wolfgang Well, no, independently from that.

Wolfgang Just to avoid to inital overhead of the function call.

How do you pass in parameters?  Consider:

def square(x):
return x*x

vs

def square(x)
while True:
yield x*x

How do you get another value of x into the generator?

 def square(x):
...   while True:
... yield x*x
...
 g = square(2)
 g
generator object at 0x3b9d28
 g.next()
4
 g.next()
4
 g.next(3)
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: expected 0 arguments, got 1

  def square(xbox):
 ... while True: yield xbox[0]*xbox[0]
 ...
  xbox = [3]
  g = square(xbox)
  g.next()
 9
  xbox[0]=4
  g.next()
 16
  [g.next() for xbox[0] in xrange(10)]
 [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

One way to answer your question literally,
whatever it may do to your gag reflex ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: customized instance dictionaries, anyone?

[Bengt Richter]

On 25 Jan 2006 09:35:50 -0800, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:

thx! indeed, it worked -- took me some time to figure out how to
implement the setting of attributes, too. i finally managed to get that
done using super:
Seems good.
snip

the ``~.__setattr__()`` method tests for the special name ``_dict`` and
defers execution to the super of the ``X`` instance, ``object``. other
stuff is handled by the instance itself.
seems a clean way to do it.

testing that with ::

x = X()
print x.__dict__[ 'foo' ]
print x.__dict__[ 'bar' ]
print x.foo
print x.bar
print x.__dict__
x.oops = 42
print x.__dict__

yields ::

bar
THIS ITEM NOT AVAILABLE
bar
THIS ITEM NOT AVAILABLE
{'foo': 'bar'}
{'foo': 'bar', 'oops': 42}

as expected.

i tried to reason *why* the usage of a property makes this particular
piece of code work, but i seemingly can't find out. anyone around who
has thoughts on that?

I had an inspiration and I think succeeded in doing what you were originally
trying to do (replace the actual instance dict with your custom dict), which
I tried to do but didn't realize at first that the __init__ setting
of x.__dict__ was really setting x.__dict__['__dict__'] not setting the initial
x.__dict__ itself. __dict__ is a peculiar animal, and looking for an instance's
attribute dict doesn't start at the instance. If you look for instance.__dict__,
it is just like looking for any other attribute, and it starts at 
type(instance).mro()[0]
looking for a descriptor (which a method also is). But the first thing found 
is
a dict proxy for looking up attributes, and when it looks up '__dict__' it 
returns
a descriptor, which then gets its __get__ method called with the instance whose 
'__dict__'
is being sought. You have to use the corresponding __set__ method to set the 
value of '__dict__'
(in this case the CustomDict instance). Otherwise instance.__dict__ will 
automatically
be set to {} and then used so you have {'__dict__':CustomDict()} instead of 
CustomDict() itself.

Once this is initialized correctly, then all the machinery works, except we 
still need to intercept
__getattr__ for some reason. I suspect this is another symptom of some 
optimization. One of these
days I will have to look in the source ;-) You would think it could give up on 
the mro method search
and get the __dict__ normally to get the attribute, but some mechanism is not 
finding the custom
dict, or else maybe it's bypassing the __getitem__ and going directly to the 
base dict method.

Someday I'll have to look in the source ;-)

customdict as before

  class CustomDict( dict ):
 ... defaultValue = 'THIS ITEM NOT AVAILABLE'
 ... def __getitem__( self, name ):
 ... try:
 ... return super( CustomDict, self ).__getitem__( name )
 ... except KeyError:
 ... return self.defaultValue
 ... def __contains__( self, name ):
 ... return True
 ... def has_key( self, name ):
 ... return True
 ...
  class X( object ):
 ... def __getattr__(self, attr):
 ... return self.__dict__[attr]
 ... #return type(self).__dict__['__dict__'].__get__(self)[attr]
 ... def __init__( self, *args, **kw ):
 ... type(self).__dict__['__dict__'].__set__(self, CustomDict(*args, 
**kw)
 ...
  x = X(foo='bar')
  print x.__dict__['foo']
 bar
  print x.__dict__['bar']
 THIS ITEM NOT AVAILABLE
  print x.foo
 bar
  print x.bar
 THIS ITEM NOT AVAILABLE
  x.oops = 42
  print x.__dict__
 {'foo': 'bar', 'oops': 42}
 
Looking at a few things of interest:

  vars(x)
 {'foo': 'bar', 'oops': 42}
  type(vars(x))
 class '__main__.CustomDict'
  type(x.__dict__)
 class '__main__.CustomDict'
  vars(x)['?']
 'THIS ITEM NOT AVAILABLE'
  type(x)
 class '__main__.X'
  type(x).__dict__
 dictproxy object at 0x02E81554
  type(x).__dict__['__dict__']
 attribute '__dict__' of 'X' objects
  type(x).__dict__['__dict__'].__get__
 method-wrapper object at 0x02EF3B6C
  type(x).__dict__['__dict__'].__get__(x)
 {'foo': 'bar', 'oops': 42}
  type(type(x).__dict__['__dict__'].__get__(x))
 class '__main__.CustomDict'

The reason the property was needed before was really several reasons.
First was that x.__dict__ wasn't being set properly, so the internal
machinery wasn't finding the custom dict. I changed the name to _dict
and let it be an ordinary attribute, but then used property to fake
what normal stuff would do if x.__dict__ itself was set, not 
x.__dict__['__dict__']

Wasted a bunch of time trying to get rid of that __getattr__ ;-/

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pulling numbers from ASCII filename not working

[Bengt Richter]

On 25 Jan 2006 12:42:20 -0800, IamIan [EMAIL PROTECTED] wrote:

Thank you for the replies, I'm new to Python and appreciate your
patience. I'm using Python 2.1.

To reiterate, the ASCII files in the workspace are being read correctly
and their latitude values (coming from the filenames) are successfully
being converted to string. Even doing LatInt = int(LatString) works,
however the second I try to print LatInt's value or use it in
mathematical operations, the code chokes in ArcGIS.

My full code:

# Import system modules
import sys, os, win32com.client

# Create the geoprocessor object
gp = win32com.client.Dispatch(esriGeoprocessing.GpDispatch.1)
print gp.usage(Hillshade_sa)
print gp.usage(RasterToOtherFormat_conversion)
print gp.usage(DefineProjection_management)

# Check license availability
gp.AddMessage (ArcInfo license is  + str(gp.CheckProduct(ArcInfo)))
gp.SetProduct(ArcInfo)
gp.CheckOutExtension(Spatial)

# Set workspace
workspace = E:\\GISTest
gp.workspace = workspace
gp.AddMessage(Workspace =  + gp.workspace)

filenames = os.listdir(gp.workspace)
filenames = [filename.lower()
for filename in filenames
if (filename[-4:].lower() == .asc and filename[0] != - )]
for filename in filenames:

   # For each ASCII file, create Hillshade.
   # account for latitude by computing Z units using radians
   Latitude = filename[1:3]
   LatString = str(Latitude)
   LatInt = int(LatString)
^^--here you set LatInt to an integer

   gp.AddMessage(LatInt is  + LatInt)
   radians = LatInt * 0.0174532925
   zFactor = 1/(113200 * (cos(radians)))

The complete traceback:

Traceback (most recent call last):
  File e:\python21\pythonwin\pywin\framework\scriptutils.py, line
310, in RunScript
exec codeObject in __main__.__dict__
  File E:\Documents and
Settings\Administrator\Desktop\Ian\GIS\Python\zOnly.py, line 32, in ?
gp.AddMessage(LatInt is  + LatInt)
   string--   ^^--integer
(LatString in place of LatInt might work, since it's a string (so is Latitude))
TypeError: cannot add type int to string
^^   ^^^ ^^
This is not lying ;-)


I tried print repr(filename) and it returned the actual filename:
'n16w099.asc' , 'n17w062.asc' , etc.
So you can see Latitude would be '16' '17' etc. right?

On to the next traceback ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pulling numbers from ASCII filename not working

[Bengt Richter]

On 24 Jan 2006 10:44:32 -0800, IamIan [EMAIL PROTECTED] wrote:

I searched the archives but couldn't find anyone else with this
problem. Basically I'm grabbing all ASCII files in a directory and
doing geoprocessing on them. I need to calculate a z-factor based on
the latitude of the ASCII file being worked on, which is in the
filename. If I type in the code manually it works and reads the
latitude value from the ASCII filename, but when run within ArcGIS it
crashes when it gets to int(LatString). Isnumber() returned false for
Latitude as well. Is there something different about reading values
from an ASCII filename?
Aren't you curious as to what the value of LatString was that failed?
Don't you know how to find out?


import sys, os, win32com.client, string, gc

# Get a list of ASCII files in the workspace for ASCII To Raster
conversion
filenames = os.listdir(gp.workspace)
filenames = [filename.lower()
for filename in filenames
if (filename[-4:].lower() == .asc and filename[0] != - )]
indentation of the above two lines would improve readability

for filename in filenames:
I would try print repr(filename) here, to see what you are dealing with

# For each ASCII file, create Hillshade.
# account for latitude by computing Z units using radians
Latitude = filename[1:3]
LatString = str(Latitude)
you probably won't need a print repr(LatString) here if you see the above print
LatInt = int(LatString)
radians = LatInt * 0.0174532925
zFactor = 1/(113200 * (cos(radians)))

BTW, capitalizing the first letter of python variable names is counter to usual 
convention.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: customized instance dictionaries, anyone?

[Bengt Richter]

On 24 Jan 2006 09:30:00 -0800, [EMAIL PROTECTED] wrote:

some time after posting my `Linkdict recipe`__ to aspn__
-- basically, a dictionary with run-time delegational
lookup, but this is not important here -- i thought gee
that would be fun to make such a customized dictionary
thingie an instance dictionary, and get some custom
namespace behavior out of that.

..  __: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/465748
..  __: http://aspn.activestate.com/

here is a simplified example: first, the customized
dictionary class and a test::


class CustomDict( dict ):

defaultValue = 'THIS ITEM NOT AVAILABLE'

def __getitem__( self, name ):
try:
return super( CustomDict, self ).__getitem__( name )
except KeyError:
return self.defaultValue

def __contains__( self, name ):
return True

def has_key( self, name ):
return True

print ''
cd = CustomDict( foo = 'bar' )
print cd[ 'foo' ]
print cd[ 'bar' ]
print 'bar' in cd
print cd.has_key( 'bar' )

this gives us::


bar
THIS ITEM NOT AVAILABLE
True
True

so it appears to work. note that we may have failed to
implement all the conceivable ways to test for
membership (such as searching through ``keys()``) or to
retrieve a value for a given key. more on that below.
now for the class to utilize this definition::

class X( object ):

def __init__( self ):
self.__dict__ = CustomDict( foo = 'bar' )

and the test code for that::

print ''
x = X()
print x.__dict__[ 'foo' ]
print x.__dict__[ 'bar' ]
print x.foo
print x.bar

which yields::


bar
THIS ITEM NOT AVAILABLE
bar

Traceback (most recent call last):
  File C:\home\projects\__svn__\sundry\#.py, line 39, in ?
print x.bar
AttributeError: 'X' object has no attribute 'bar'

ok. so the custom dict *basically* works as expected,
since it does successfully make ``x.foo`` available --
no surprise here. unfortunately, looking up ``x.bar``,
which should really return the default value string,
causes an ``AttributeError`` to be raised.

now of course given the short definition of
``CustomDict``, perhaps there is an essential lookup
method that has not been overwritten but that is
internally used for attribute lookup. however, i
carefully tested my actual class (from the recipe
mentioned above) and also compared the methods defined
there against the standard ``dict()`` interface, and
nothing of importance appeared to be missing. i also
tried to bind the dictionary to the instance earlier, in
``__new__``, to no avail. am i missing something here?

Well, if you compare with the following, maybe something will fall into place?

  class CustomDict( dict ):
 ... defaultValue = 'THIS ITEM NOT AVAILABLE'
 ... def __getitem__( self, name ):
 ... try:
 ... return super( CustomDict, self ).__getitem__( name )
 ... except KeyError:
 ... return self.defaultValue
 ... def __contains__( self, name ):
 ... return True
 ... def has_key( self, name ):
 ... return True
 ...
  class X( object ):
 ... __dict__ = property(lambda self:self._dict)
 ... def __getattr__(self, attr): return self.__dict__[attr]
 ... def __init__( self ):
 ... self._dict = CustomDict( foo = 'bar' )
 ...
  x = X()
  print x.__dict__['foo']
 bar
  print x.__dict__['bar']
 THIS ITEM NOT AVAILABLE
  print x.foo
 bar
  print x.bar
 THIS ITEM NOT AVAILABLE

Additional data points:
  x.__dict__
 {'foo': 'bar'}
  X.__dict__
 dictproxy object at 0x02E814C4
  X.__dict__['__dict__']
 property object at 0x02EEF70C

and

  class Y(object):
 ... def _getdict(self): print '_getdict'; return self._dict
 ... __dict__=property(_getdict)
 ... def __init__( self ):
 ... self._dict = CustomDict( foo = 'bar' )
 ...
  y = Y()
  y.__dict__
 _getdict
 {'foo': 'bar'}
  y._dict
 {'foo': 'bar'}
  y.foo
 Traceback (most recent call last):
   File stdin, line 1, in ?
 AttributeError: 'Y' object has no attribute 'foo'
  def ga(self, attr): print '__getattr__(%s)'%attr; return 
  self.__dict__[attr]
 ...
  Y.__getattr__ = ga
  y.foo
 __getattr__(foo)
 _getdict
 'bar'

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python code written in 1998, how to improve/change it?

[Bengt Richter]

On Tue, 24 Jan 2006 14:58:27 -0600, [EMAIL PROTECTED] wrote:


Wolfgang So basically if I want to write a long-running program in
Wolfgang Python, it would make sense to code all functions that are
Wolfgang likely to be called more than once as generators...

If they need to resume their calculations from where they left off after the
last yield.
Hm, I wonder how (all untested)

def foo(x,y):
return x**2+y**2
for pair in pairs: print foo(*pair)

would compare to

def bar(arglist):
while True:
x,y = arglist
yield x**2 + y**2
barargs = []
ibar = bar(barargs).next
for barargs[:] in pairs: print ibar()

Of course, for simple stuff there's always manual inlining ;-)

for x, y in pairs: print x**2, y**2

Hm, bf warning it might be interesting if one could bind arg list items of
a generator from the outside, e.g.,

def gfoo(x, y):
while True:
   yield x**2 + y**2

ifoo = gfoo('dummy','dummy') # or first pair
for ifoo.x, ifoo.y in pairs: print ifoo.next()

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using non-ascii symbols

[Bengt Richter]

On Tue, 24 Jan 2006 04:09:00 +0100, Christoph Zwerschke [EMAIL PROTECTED] 
wrote:

On the page http://wiki.python.org/moin/Python3%2e0Suggestions
I noticed an interesting suggestion:

These operators ≤ ≥ ≠ should be added to the language having the 
following meaning:

   = = !=

this should improve readibility (and make language more accessible to 
beginners).

This should be an evolution similar to the digraphe and trigraph 
(digramme et trigramme) from C and C++ languages.

How do people on this group feel about this suggestion?

The symbols above are not even latin-1, you need utf-8.

Maybe we need a Python unisource type which is abstract like unicode,
and through encoding can be rendered various ways. Of course it would have
internal representation in some encoding, probably utf-16le, but glyphs
for operators and such would be normalized, and then could be rendered
as multi-glyphs or special characters however desired. This means that
unisource would not just be an encoding resulting from decoding just
a character encoding like latin-1, but would be a result of decoding
source in a Python-syntax-sensitive way, differentiating between =
as a relational operator vs '=' in a string literal or comment etc.

(There are not many usefuls symbols in latin-1. Maybe one could use × 
for cartesian products...)

And while they are better readable, they are not better typable (at 
least with most current editors).

Is this idea absurd or will one day our children think that restricting 
to 7-bit ascii was absurd?
I think it's important to have readable ascii representations available for
programming elements at least.

Are there similar attempts in other languages? I can only think of APL, 
but that was a long time ago.

Once you open your mind for using non-ascii symbols, I'm sure one can 
find a bunch of useful applications. Variable names could be allowed to 
be non-ascii, as in XML. Think class names in Arabian... Or you could 
use Greek letters if you run out of one-letter variable names, just as 
Mathematicians do. Would this be desirable or rather a horror scenario? 
Opinions?
I think there are pros and cons. What if the href in HTML could be spelled in
any characters? I.e., some things are part of a standard encoding and 
representation
system. Some of python is like that. True should not be spelled Vrai or 
Sant, except
in localized messages, IMO, unless perhaps there is a unisource type that 
normalizes
these things too, and can render in localized formats. ... I guess China is a
pretty big market, so I wonder what they will do.

Someone has to get really excited about it, and have the expertise or 
willingness
to slog their way to expertise, and the persistence to get something done. And 
all
that in the face of the fact that much of the problem will be engineering 
consensus,
not engineering technical solutions. So are you excited? Good luck ;-)

Probably the best anyone with any excitement to spare could do is ask Martin
what he could use help with, if anything. He'd probably not like muddying any
existing clear visions and plans with impractical ramblings though ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird generator id() behaviour (was Re: Python code written in1998, howto improve/change it?)

[Bengt Richter]

On Wed, 25 Jan 2006 12:33:17 +1300, Carl Cerecke [EMAIL PROTECTED] wrote:

Adding a continue statemtent after the yield statements yields :-) a 
speed increase. Still not as good as functions though. (about 30% slower)

Cheers,
Carl

Carl Cerecke wrote:
 Carl Cerecke wrote:
 Generator FSM done properly (well, better anyway). They are still almost 
 twice as slow as plain function calls, though.
 
snip
I think I would use a generator to do state transitions, but make the state
external, or at least the part that's interesting to the outside world.

In this peculiar example the transition rules are the same for both states,
so you only need one implementation of the logic. So the example is not so nice.


  def fsm(state, events):
 ... for action in events:
 ... if action == 'lift': state.name = 'ON'
 ... elif action == 'push': state.name = 'OFF'
 ... else:
 ... state.name = 'END'
 ... break
 ... yield state
 ...
  def actions(n):
 ... import random
 ... return iter([random.choice(['lift','push']) for i in range(n-1)] + 
[None])
 ...
  class State(object): pass
 ...
  def test(r=100):
 ... state = State()
 ... state.name = 'ON'
 ... from time import clock
 ... t0 = clock()
 ... for state in fsm(state, actions(r)): pass
 ... t1 = clock()
 ... print '%12.6f'%((t1-t0)/r)
 ...
  test(1000)
 0.58
  test(1000)
 0.32
  test(1000)
 0.32
  test(10)
 0.32
  a = list(actions(10))
  a
 ['lift', 'push', 'push', 'lift', 'push', 'lift', 'push', 'lift', 'lift', None]
  state = State()
  state.name = 'START'
  f = fsm(state, a)
  for state in f: print state.name,
 ...
 ON OFF OFF ON OFF ON OFF ON ON
 
 
Obviously you can keep state in the fsm generator by placing yields in 
different places and
looping in different ways, but always storing the externally interesting state 
as attributes
of the state parameter and yielding that to tell the world the latest. Since 
only attributes
are being modified, the original state binding could be used and the 
generator's yielded
value could be ignored, but it could be handy if the generator is passed around 
IWT.

The world could also feed info in as attributes of state. And other generators 
could share
the same external state variable and all kinds of weird things could be built 
;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Redirecting standard out in a single namespace

[Bengt Richter]

On 23 Jan 2006 04:00:40 -0800, Fuzzyman [EMAIL PROTECTED] wrote:


Bengt Richter wrote:

[...]
 It wouldn't be shadowing, but I suppose you could replace sys.stdout with
 a custom object whose methods check where they were called from.
 Then you could give the object initialization parameters as to which 
 namespace
 you want to have the effect in, and/or methods to control the action or turn
 it on or off etc. BTW, how about stderr?


Redirecting stderr is identical in concept to redirecting stdout.

Yeah, just reminding, in case you need to do that too ;-)

The following in the write method of the custom out object works :

sys._getframe(1).f_globals['__name__']

sys.stdout.write is *always* called from at least one frame deep in the
stack - so it works.
Yes, that's about what I had in mind, not being able to think of an alternative
(other than the obvious one of using something other than print in the module
where you want to redirect, either by hand or by effective source rewrite one 
way
or another to translate print statments into e.g. myprint('the', 'print', 
'args') or such.
You could do source preprocessing or translating at the AST level, or you could 
do byte code munging.
Nothing too nice.


However the Python docs say of sys._getframe :

This function should be used for internal and specialized purposes
only.

Might this approach be brittle ?

In what sense? I guess it ought to work for the version it's defined for, but 
the advice is there.
Maybe we can get something more specific from the author of the above doc 
snippet ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Decimal vs float

[Bengt Richter]

On Sat, 21 Jan 2006 14:28:20 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote:

On Fri, 20 Jan 2006 04:25:01 +, Bengt Richter wrote:

 On Thu, 19 Jan 2006 12:16:22 +0100, =?ISO-8859-1?Q?Gerhard_H=E4ring?= 
 [EMAIL PROTECTED] wrote:
 [...]

floating points are always imprecise, so you wouldn't want them as an 
 Please, floating point is not always imprecise. In a double there are
 64 bits, and most patterns represent exact rational values. Other than
 infinities and NaNs, you can't pick a bit pattern that doesn't have
 a precise, exact rational value. 

Of course every float has a precise rational value.
0.11 has a precise rational value:

11/100

Good, I'm glad that part is clear ;-)

But that's hardly what people are referring to. The question isn't whether
people?
every float is an (ugly) rational, but whether every (tidy) rational is a
float. And that is *not* the case, simple rationals like 1/10 cannot be
written precisely as floats no matter how many bits you use. 
See the next statement below. What did you think I meant?

 You can't represent all arbitarily chosen reals exactly as floats, that's 
 true,
 but that's not the same as saying that floating points are always 
 imprecise.

Always is too strong, since (for example) 1/2 can be represented
precisely as a float. But in general, for any random rational value N/M,
the odds are that it cannot be represented precisely as a float. And
that's what people mean when they say floats are imprecise.
That's what *you* mean, I take it ;-) I suspect what most people mean is that 
they don't
really understand how floating point works in detail, and they'd rather not 
think about it
if they can substitute a simple generalization that mostly keeps them out of 
trouble ;-)

Besides, cannot be represented precisely is a little more subtle than numbers 
of bits.
E.g., one could ask, how does the internal floating point bit pattern for 
0.10001
(which incidentally is not the actual exact decimal value of the IEEE 754 bit 
pattern --
0.155511151231257827021181583404541015625 is the exact value)
*not* represent 0.1 precisely? E.g., if all you are interested in is one 
decimal fractional
digit, any float whose exact rational value is f where .05 = f  0.15 could be 
viewed as one
in a (quite large) set of peculiar error-correcting codes that all map to the 
exact value you
want to represent. This is a matter of what you mean by represent vs what is 
represented.
Float representations are just codes made of bits. If what you want is for 
'%5.2f'%f to
produce two reliably exact decimal fractional digits, you have a lot of choices 
for f. Chances
are f = 0.1 won't make for a surprise, which in some sense means that the float 
bits behind float('.1')
represented .1 exactly, even though they did so by way of an unambiguously 
associated nearby
but different mathematically exact value.

BTW, equally important to precision of individual numbers IMO is what happens 
to the precision of
results of operations on inexactly represented values. How do errors accumulate 
and eventually
cause purportedly precise results to differ from mathematically exact results 
more than the advertised
precision would seem to allow? This kind of question leads to laws about when 
and how to round,
and definition of legal usage for factors e.g. converting from one currency to 
another, where
the inverse conversion factor is not a mathematical inverse.

Practicality is beating up on purity all over the place ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Redirecting standard out in a single namespace

[Bengt Richter]

On 20 Jan 2006 07:37:15 -0800, Fuzzyman [EMAIL PROTECTED] wrote:

Hello,

I'm trying to redirect standard out in a single namespace.

I can replace sys.stdout with a custom object - but that affects all
namespaces.

There will be code running simultaneously that could import sys
afterwards - so I don't want to make the change in the sys module.

I have an idea to redefine __import__ for the relevant namespace - so
that an attempt to import sys will return a different module with a
custom object for stdout. As sys is a builtin module this might not
work for the print statement, which is what I want to redirect.

I'm not in a position to test my idea until later :

__import = __import__
def __import__(name):
if name == 'sys':
return __import('newsys')
else:
return __import(name)

import sys

Is there another way to shadow the sys module from a single namespace ?

It wouldn't be shadowing, but I suppose you could replace sys.stdout with
a custom object whose methods check where they were called from.
Then you could give the object initialization parameters as to which namespace
you want to have the effect in, and/or methods to control the action or turn
it on or off etc. BTW, how about stderr?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python code written in 1998, how to improve/change it?

[Bengt Richter]

On Mon, 23 Jan 2006 08:53:59 +1300, Carl Cerecke [EMAIL PROTECTED] wrote:

Bengt Richter wrote:
 On Thu, 19 Jan 2006 23:16:57 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

 How about something like
 
   actions = dict(
  ...a=compile('print A; state=b','','exec'),
  ...b=compile('print B; state=c','','exec'),
  ...c=compile('print C; state=None','','exec')
  ... )
   state = 'a'
   while state: eval(actions[state])
  ...
  A
  B
  C

Good idea. But we can eliminate the dictionary lookup:

a1 = compile('print A; state=b1','','exec')
b1 = compile('print B; state=c1','','exec')
c1 = compile('print C; state=None','','exec')

state = a1
while state:
 eval(state)

Cool. But unfortunately, neither version works inside a function's local 
namespace.
Using exec instead of eval seems to do it in either context though.
Now, how can we get optimized code (i.e., LOAD_FAST vs LOAD_NAME etc in a1 etc)
without byte code hackery?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Arithmetic sequences in Python

[Bengt Richter]

On Mon, 23 Jan 2006 21:43:16 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote:

On Sun, 22 Jan 2006 16:40:48 -0800, Paul Rubin wrote:

 Steve Holden [EMAIL PROTECTED] writes:
  The current list function is supposed to be something like a
  typecast:
 
 list() isn't a function, it's a type.
 
 I'm not sure what the distinction is supposed to be.  list is anyway
 callable, and lambda a:list(a) is certainly a function. 


class Parrot:
def __init__(self):
pass

Parrot is callable. Is it a function?

No. It is an object that inherits a __call__ method that makes it callable with 
a (args) source syntax trailer
in a similar way to an object created with a def (which is a function in python 
convention, and which
also inherits a __call__ method), but the similarity does not make Parrot a 
function:

  class Parrot:
 ... def __init__(self):
 ... pass
 ...
(BTW you could have inherited a do-nothing __init__ ;-)

  type(Parrot).mro()
 [type 'classobj', type 'object']
  type(Parrot).mro()[0].__call__
 slot wrapper '__call__' of 'classobj' objects
Or, showing more explicitly where it comes from:
  type(Parrot).mro()[0].__dict__['__call__']
 slot wrapper '__call__' of 'classobj' objects

Invoked:
  type(Parrot).mro()[0].__dict__['__call__'](Parrot)
 __main__.Parrot instance at 0x02EF340C

Hm, actually that is like calling the im_func of the unbound method of a 
newstyle class ...
I think maybe actually Parrot() causes

  type(Parrot).mro()[0].__dict__['__call__'].__get__(Parrot, type(Parrot))()
 __main__.Parrot instance at 0x02EF398C

A function is also an object with a __call__ method. E.g., compare above the 
line with
calling foo (after definition just below) the long way:

  def foo(): return 'returned by foo'
 ...
  type(foo).mro()[0].__dict__['__call__'].__get__(foo, type(foo))()
 'returned by foo'

Playing with that a little:
  type(foo).mro()
 [type 'function', type 'object']
  type(foo).mro()[0]
 type 'function'
  type(foo).mro()[0].__call__
 slot wrapper '__call__' of 'function' objects
  type(foo).mro()[0].__call__(foo)
 'returned by foo'
  foo.__call__
 method-wrapper object at 0x02EF340C
  foo.__call__()
 'returned by foo'
  type(foo).mro()[0].__call__.__get__
 method-wrapper object at 0x02EF340C
  type(foo).mro()[0].__call__.__get__(foo, type(foo))
 method-wrapper object at 0x02EF39AC
  type(foo).mro()[0].__call__.__get__(foo, type(foo))()
 'returned by foo'
or
  foo()
 'returned by foo'


Types are types, classes are classes, functions are functions.
classes seem to be classobjs, designed to implement classic class behavior
but using the new machinery to achieve compatible integration.


Admittedly I still confused between the various flavours of functions
(function, bound method, unbound method, class method, static method...)
*wink* but the difference between types and functions is fairly clear.

Just don't ask about the difference between type and class... *wink*

Why not? ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python code written in 1998, how to improve/change it?

[Bengt Richter]

On Thu, 19 Jan 2006 23:16:57 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

Carl Cerecke wrote:
 Carl Cerecke wrote:
Ah. Well, my post suggested, as one option, the callables call
each other directly.
 
 Doh! No I didn't. And they shouldn't. Otherwise the call stack
 gets out of hand. But I did suggest that each callable representing a 
 state set a global variable, just before it returns, to the callable 
 representing the next state to be called. Still no map required. Just a 
 while loop. In any case, the function call/return is wasted cycles.

I believe the more modern approach to this is to use generators in some 
way, yield each other as the next state.  This way you avoid all almost 
all the function call overhead (the part that takes significant time, 
which is setting up the stack frame) and don't have to resort to 
bytecode hacks for better performance.

Of course, if you have a state machine with many small states each doing 
a tiny bit of processing and you're still concerned over performance, 
you probably should be looking into Pysco or Pyrex and avoid making your 
code really unreadable.

How about something like

  actions = dict(
 ...a=compile('print A; state=b','','exec'),
 ...b=compile('print B; state=c','','exec'),
 ...c=compile('print C; state=None','','exec')
 ... )
  state = 'a'
  while state: eval(actions[state])
 ...
 A
 B
 C

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: OT: excellent book on information theory

[Bengt Richter]

On Thu, 19 Jan 2006 14:12:24 +0200, Juho Schultz [EMAIL PROTECTED] wrote:

Anton Vredegoor wrote:
 
 Returning to the original book, why did they write a lot of it (at
 least the first few pages until I gave up, after having trouble
 understanding formulas about concepts I have no such trouble with when
 framed in less jargonized from) in unintelligible mathemathical
 notation when there's Python?
 

Because the intended audience is probably reads formulas better than
they read Python. The 1st sentence of the Introduction: This book is 
aimed at senior undergraduates and graduate students in Engineering, 
Science, Mathematics and Computing.

Last month I spent about an hour trying to explain why
a*2.5e-8 = x
raises a SyntaxError and why it should be written
x = a*2.5e-8
The guy who wrote the 1st line has MSc in Physics from Cambridge (UK).
In mathematics, there is no difference between the two lines.

ISTM probable that his original equation was really saying
assert a*2.5e-8 == x
which is not very different from
assert x == a*2.5e-8

Did you mention that = is not == in python?
I too would resist the idea that
assert a*2.5e-8 == x
should be written as
x = a*2.5e-8

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Efficient implementation of deeply nested lists

[Bengt Richter]

On 19 Jan 2006 01:19:06 -0800, Kay Schluehr [EMAIL PROTECTED] wrote:

I want to manipulate a deeply nested list in a generic way at a
determined place. Given a sequence of constant objects a1, a2, ..., aN
and a variable x. Now construct a list from them recursively:

L = [a1, [a2, [[aN, [x]]...]]

The value of x is the only one to be changed. With each value of x a
new instance of L is needed so that we actually have a function:

L = lambda x: [a1, [a2, [[aN, [x]]...]]

I'm seeking for an efficient implementation that does not recompute the
whole list each time. Any ideas?

Make a custom class factory that makes a class with a1..aN and can be 
instantiated
with x, producing an object that behaves like L

So the question in my mind is, how are you actually going to use these L 
things?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Decimal vs float

[Bengt Richter]

On Thu, 19 Jan 2006 12:16:22 +0100, =?ISO-8859-1?Q?Gerhard_H=E4ring?= [EMAIL 
PROTECTED] wrote:
[...]

floating points are always imprecise, so you wouldn't want them as an 
Please, floating point is not always imprecise. In a double there are
64 bits, and most patterns represent exact rational values. Other than
infinities and NaNs, you can't pick a bit pattern that doesn't have
a precise, exact rational value. BTW, you'd need a 64-bit CPU to get 
range(-2**53,2**53+1)
but the 53 bits of available precision a float (IEEE 754 double) can represent
each integer in that range exactly (and of course similar sets counting by 2 or 
4 etc.)

You can't represent all arbitarily chosen reals exactly as floats, that's true,
but that's not the same as saying that floating points are always imprecise.

As a practical matter it is hard to track when floating point calculations lose
exactness (though UIAM there are IEEE 754 hardware features that can support 
that),
so it is just easier to declare all floating point values to be tainted with 
inexactness
from the start, even though it isn't so.

1.0 is precisely represented as a float. So is 1.5 and so are more other values 
than
you can count with an ordinary int ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

[OT] Nit: please don't user it's unless you can substitute it is without changing your inteded meaning.

[Bengt Richter]

Typos happen to all of us, but in case you hadn't realized what it's
is a contraction for (it is), now you do, and you can save yourself further
embarrassment (assuming you care ;-).
If your friends won't tell you, who will ;-)

Write the contraction it's only when you want its meaning
to be it is, otherwise it's wrong ;-)

OTOH, how does one punctuate the posessive of a word per se?
E.g., the first letter of it is i, but can one write that
as it's first letter is i, or it's first letter is i ? ;-)
And how many its (?) are there in the previous sentence?

I wonder if Eats Leaves and Shoots (a book on punctuation) has something on 
that.
(vs, Eats, Leaves, and Shoots -- panda vs gunslinger).

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nit: please don't user it's unless you can substitute it is without changing your inteded meaning.

[Bengt Richter]

On 18 Jan 2006 04:19:37 -0800, Fuzzyman [EMAIL PROTECTED] wrote:


Bengt Richter wrote:
 Typos happen to all of us, but in case you hadn't realized what it's
 is a contraction for (it is), now you do, and you can save yourself further
 embarrassment (assuming you care ;-).
 If your friends won't tell you, who will ;-)


Making grammatical errors in the subject of a post on gramamtical
errors must be embarrasing. ;-)
Yeah. I could probably get away with claiming that I inteded one error to 
soften the
blow of my demonstration of superiority, but actually neither was intentional.
Perhaps it worked out for the best ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Preventing class methods from being defined

[Bengt Richter]

On Mon, 16 Jan 2006 18:55:43 -0800, David Hirschfield [EMAIL PROTECTED] wrote:

Thanks for this, it's a great list of the ways it can be done. Here's a 
Actually, your way is yet another ;-)

bit more insight into the arrangement I'm trying to get:

restrict = True
Why a global value? If it is to affect class instantiation, why not pass it
or a value to the constructor, e.g., C(True) or C(some_bool)?


class A(object):
   ^--should that be R?
_restrict = [test]
   
def _null(self, *args, **kws):
raise Exception,not allowed to access
   
def test(self):
print test restricted
   
def __init__(self):
if restrict:
for f in self._restrict:
setattr(self,f,self._null)
I assume you know that you are using a bound method attribute
on the instance to shadow the method of the class, for a per-instance
effect as opposed to an all-instances shared effect.

   
class C(R):
def __init__(self):
super(C,self).__init__()
   
def test(self):
print test from c


In this design, calling c.test() where c is an instance of C will raise 
an exception. Now, the only thing I'd like is to not have to fill out 
that _restrict list like that, but to have some function or something 
that let's me say which methods are restricted in the same way you 
define class methods or properties, i.e.:

class A(object):
_restrict = []
   
def _null(self, *args, **kws):
raise Exception,not allowed to access
   
def test(self):
print test restricted
restrict(test)
 this does some magic to insert test into the _restrict list


I can't really find a way to make that work with descriptors, and it 
can't just be a function call, because I won't know what object to get 
the _restrict list from. Is there a way to refer to the class that is 
being defined when calling a function or classmethod?
So, ideas on how to accomplish that...again, greatly appreciated.

You can do it with a decorator, though it doesn't really do decoration,
just adding the decoratee to the associated _restrict list. You don't
have to factor out mkrdeco if you'r'e only defining the restrict decorator
in one class.

I changed A to R, and made the global restriction flag a constructor argument,
but you can easily change that back, either by using the global restricted
in R.__init__ as a global, or by passing it explicitly like c = C(restricted).

  def mkrdeco(rlist):
 ... def restrict(f):
 ... rlist.append(f.func_name)
 ... return f
 ... return restrict
 ...
  class R(object):
 ... _restrict = []
 ... restrict = mkrdeco(_restrict)
 ... def _null(self, *args, **kws):
 ... raise Exception,not allowed to access
 ... def __init__(self, restricted):
 ... if restricted:
 ... for f in self._restrict:
 ... setattr(self,f,self._null)
 ... @restrict
 ... def test(self):
 ... print test restricted
 ...
  class C(R):
 ...def __init__(self, restricted=False):
 ...super(C,self).__init__(restricted)
 ...
 ...def test(self):
 ...print test from c
 ...
  c = C(True)
  c.test()
 Traceback (most recent call last):
   File stdin, line 1, in ?
   File stdin, line 5, in _null
 Exception: not allowed to access
  c2 = C(False)
  c2.test()
 test from c
  vars(c)
 {'test': bound method C._null of __main__.C object at 0x02EF3C4C}
  vars(c2)
 {}
  R._restrict
 ['test']
 
Still don't know what real application problem this is solving, but that's ok 
;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Decimal ROUND_HALF_EVEN Default

[Bengt Richter]

On 16 Jan 2006 20:36:12 -0800, Raymond Hettinger [EMAIL PROTECTED] wrote:

LordLaraby wrote:
 If 'bankers rounding' is HALF_ROUND_EVEN, what is HALF_ROUND_UP? I
 confess to never having heard the terms.

The terms are defined in the docs for the Context object:
http://docs.python.org/lib/decimal-decimal.html

The rounding option is one of:
--
ROUND_CEILING (towards Infinity),
ROUND_DOWN (towards zero),
ROUND_FLOOR (towards -Infinity),
ROUND_HALF_DOWN (to nearest with ties going towards zero),
ROUND_HALF_EVEN (to nearest with ties going to nearest even integer),
ROUND_HALF_UP (to nearest with ties going away from zero), or
ROUND_UP (away from zero).


 I usually do: Y = int(X + 0.5) scaled to proper # of decimal places.
 Which type of rounding is this? If either.

The interpreter shows that ties are rounded down towards zero:

 [int(x + 0.5) for x in range(-5, 6)]
[-4, -3, -2, -1, 0, 0, 1, 2, 3, 4, 5]

Or more explicitly:
  for x in xrange(-5,6): print 'int(%2s + 0.5) == int(%4s) = 
  %2s'%(x,x+.5,int(x+.5))
 ...
 int(-5 + 0.5) == int(-4.5) = -4
 int(-4 + 0.5) == int(-3.5) = -3
 int(-3 + 0.5) == int(-2.5) = -2
 int(-2 + 0.5) == int(-1.5) = -1
 int(-1 + 0.5) == int(-0.5) =  0
 int( 0 + 0.5) == int( 0.5) =  0
 int( 1 + 0.5) == int( 1.5) =  1
 int( 2 + 0.5) == int( 2.5) =  2
 int( 3 + 0.5) == int( 3.5) =  3
 int( 4 + 0.5) == int( 4.5) =  4
 int( 5 + 0.5) == int( 5.5) =  5

So one must be careful not to use int to convert relative screen positions to 
relative pixel deltas
to add to some pixel screen position, even when it's a single add that can't 
accumulate error or
round differentely after the add.

I guess one would want
  for x in xrange(-5,6): print 'int(math.floor(%4s)) = 
  %2s'%(x+.5,int(math.floor(x+.5)))
 ...
 int(math.floor(-4.5)) = -5
 int(math.floor(-3.5)) = -4
 int(math.floor(-2.5)) = -3
 int(math.floor(-1.5)) = -2
 int(math.floor(-0.5)) = -1
 int(math.floor( 0.5)) =  0
 int(math.floor( 1.5)) =  1
 int(math.floor( 2.5)) =  2
 int(math.floor( 3.5)) =  3
 int(math.floor( 4.5)) =  4
 int(math.floor( 5.5)) =  5

Which ISTM is the way some other int conversions have worked, but I can't 
recall the context ATM.
I.e., just trim the fractional bits from the theoretical fixed point twos 
complement number, which
always subtracts those positive-value fractional bits algebraically.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: proposal: another file iterator

[Bengt Richter]

On 15 Jan 2006 18:58:53 -0800, Paul Rubin http://[EMAIL PROTECTED] wrote:

Jean-Paul Calderone [EMAIL PROTECTED] writes:
 Which is only very slightly longer than your version.  I would like
 it even more if iter() had been written with the impending doom of
 lambda in mind, so that this would work:
 
 for chunk in iter('', f.read, blocksize):
 ...
 
 But it's a bit late now.

Well, iter could be extended to take *args and **kwargs parameters, so
you'd say

for chunk in iter(f.read, '', (blocksize,)): ...

Whatever Which refers to got snipped or may be in a post that hasn't
become visible for me yet, but I assume Jean-Paul was referring to lambda use
as in e.g. (untested):

 for chunk in iter(lambda frd=open('yerf', 'rb').read:frd(blocksize), ''): 
...

Does it not do what you want?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: instance attributes not inherited?

[Bengt Richter]

 descriptors, so you may want
to read about it ;-)

[1] If we wanted to, we could write our own simple_super. E.g.,

  class simple_super(object):
 ... def __init__(self, inst): self.inst = inst
 ... def __getattribute__(self, attr):
 ... inst = object.__getattribute__(self, 'inst')
 ... try: return getattr(type(inst).mro()[1], attr).im_func.__get__(
 ... inst, type(inst))
 ... except AttributeError:
 ... return object.__getattribute__(self, attr)
 ...
  class Parent( object ):
 ... def __init__( self ):
 ... self.x = 9
 ... print 'Inside Parent.__init__()'
 ...
  class Child( Parent ):
 ... def __init__( self ):
 ... sup = simple_super(self)
 ... sup.__init__()
 ... self.sup = sup
 ... print Inside Child.__init__()
 ...
  c = Child()
 Inside Parent.__init__()
 Inside Child.__init__()
  c.sup
 __main__.simple_super object at 0x02EF80EC
  c.sup.__init__
 bound method Child.__init__ of __main__.Child object at 0x02EF80CC
  c.sup.__init__()
 Inside Parent.__init__()

I don't know if this helps ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is 'everything' a refrence or isn't it?

[Bengt Richter]

On Mon, 16 Jan 2006 21:58:26 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote:

On Mon, 16 Jan 2006 10:34:40 +, Bengt Richter wrote:

   class A:
  ... def __getattr__(self, attr): print 'A().%s'%attr; raise 
 AttributeError
  ...
   class B:
  ... def __getattr__(self, attr): print 'B().%s'%attr; raise 
 AttributeError
  ...
   A()==B()
  A().__eq__
  B().__eq__
  B().__eq__
  A().__eq__
  A().__coerce__
  B().__coerce__
  A().__cmp__
  B().__cmp__
  False

Why are A().__eq__ and B().__eq__ both being called twice?

I imagine it's trying the calls with args swapped or something like that.
Want to write a test and see? (Don't hold your breath if you're waiting for me 
to do it ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is 'everything' a refrence or isn't it?

[Bengt Richter]

On Mon, 16 Jan 2006 21:58:26 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote:

On Mon, 16 Jan 2006 10:34:40 +, Bengt Richter wrote:

   class A:
  ... def __getattr__(self, attr): print 'A().%s'%attr; raise 
 AttributeError
  ...
   class B:
  ... def __getattr__(self, attr): print 'B().%s'%attr; raise 
 AttributeError
  ...
   A()==B()
  A().__eq__
  B().__eq__
  B().__eq__
  A().__eq__
  A().__coerce__
  B().__coerce__
  A().__cmp__
  B().__cmp__
  False

Why are A().__eq__ and B().__eq__ both being called twice?

Looks like it has to do with trying stuff with arguments switched?
This shows call and argumenst to method for whichever attribute access
is allowed to succeed after a count of failures...

  def test():
 ... global trial, which
 ... for which in xrange(9):
 ... print '\nwhich: %s'%which
 ... trial = 0
 ... A()==B()
 ...
  def nameof(x):return type(x).__getattribute__(x, '__class__').__name__
 ...
  def __eq__(x, y):
 ... print '__eq__(%s(), %s())'%(nameof(x), nameof(y)); return False
 ...
  def __cmp__(x, y):
 ... print '__cmp__(%s(), %s())'%(nameof(x), nameof(y)); return 0
 ...
  def __coerce__(x, y):
 ... print '__coerce__(%s(), %s())'%(nameof(x), nameof(y)); return None
 ...
  class A:
 ... def __getattr__(self, attr):
 ... global trial
 ... print '%s: A().%s'%(trial, attr)
 ... if trial!=which: trial+=1; raise AttributeError
 ... return globals()[attr].__get__(self, type(self))
 ...
  class B:
 ... def __getattr__(self, attr):
 ... global trial
 ... print '%s: B().%s'%(trial, attr)
 ... if trial!=which: trial+=1; raise AttributeError
 ... return globals()[attr].__get__(self, type(self))
 ...
  test()
 
 which: 0
 0: A().__eq__
 __eq__(A(), B())
 
 which: 1
 0: A().__eq__
 1: B().__eq__
 __eq__(B(), A())
 
 which: 2
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 __eq__(B(), A())
 
 which: 3
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 3: A().__eq__
 __eq__(A(), B())
 
 which: 4
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 3: A().__eq__
 4: A().__coerce__
 __coerce__(A(), B())
 4: B().__coerce__
 __coerce__(B(), A())
 4: A().__cmp__
 __cmp__(A(), B())
 
 which: 5
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 3: A().__eq__
 4: A().__coerce__
 5: B().__coerce__
 __coerce__(B(), A())
 5: A().__cmp__
 __cmp__(A(), B())
 
 which: 6
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 3: A().__eq__
 4: A().__coerce__
 5: B().__coerce__
 6: A().__cmp__
 __cmp__(A(), B())
 
 which: 7
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 3: A().__eq__
 4: A().__coerce__
 5: B().__coerce__
 6: A().__cmp__
 7: B().__cmp__
 __cmp__(B(), A())
 
 which: 8
 0: A().__eq__
 1: B().__eq__
 2: B().__eq__
 3: A().__eq__
 4: A().__coerce__
 5: B().__coerce__
 6: A().__cmp__
 7: B().__cmp__
 
 
Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Preventing class methods from being defined

[Bengt Richter]

On Sun, 15 Jan 2006 19:23:30 -0800, David Hirschfield [EMAIL PROTECTED] wrote:

I should have explicitly mentioned that I didn't want this particular 
solution, for a number of silly reasons.
Is there another way to make this work, without needing to place an 
explicit if allowed around each method definition?

Seems like you can
a. define the class with all methods defined within, and use a metaclass to 
prune
   out the ones you don't want, which Alex provided.
b. define the class with conditional execution of the method definitions, which 
you just rejected.
c. define the class with no iffy methods at all, and add them afterwards
   c1. in a metaclass that adds them and possibly also defines them for that 
purpose
   c2. by plain statements adding method functions as class attributes
d. define all the methods normally, but monitor attribute access on the class 
and raise
   attribute error for the methods that aren't supposed to be there.
e. raise an exception conditionally _within_ methods that aren't supposed to be 
there, if called.

What would you like?

BTW, defining the method functions someplace other than within the body of the 
class whose methods
they are to become has some subtleties, since the functions can potentially 
refer to different
global scopes than that of the class (e.g. if you take the functions from an 
imported module)
and/or use closure-defined cell variables (e.g. if the method function is 
defined within a factory function).
This can be used to advantage sometimes, but needs good documentation to be 
clear for the next code maintainer ;-)

I guess I should re-read your original requirements that led to thes design 
ideas.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Listing partitions (on win32)

[Bengt Richter]

On 14 Jan 2006 16:52:33 -0800, Claude Henchoz [EMAIL PROTECTED] wrote:

Hi

Is there any way of listing partitions on a (win32) computer without
using WMI?

Maybe this will work (I skipped A: and B:, but you can include them if
you want to catch floppy drives with something in them). The 'xxx' is just so as
not to get a full directory's worth of useless text. If xxx happens to be 
defined
it doesn't hurt anything. It's not going to be as fast as using one of the
win32 api packages to get at GetLogicalDriveStrings, but this works on my NT4:
(error messages apparently go to stderr, so stdout gets '' which makes the if 
fail)

  def fakeGetLogicalDriveStrings():
 ... return [c+':' for c in (chr(n) for n in xrange(ord('A'), ord('Z')+1))
 ...   if os.popen('dir %s:\\xxx'%c).read()]
 ...
  fakeGetLogicalDriveStrings()
 ['C:', 'D:', 'E:', 'V:', 'W:']

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Listing partitions (on win32)

[Bengt Richter]

On Sun, 15 Jan 2006 08:08:16 -0500, Roger Upole [EMAIL PROTECTED] wrote:


Bengt Richter [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED]
 On 14 Jan 2006 16:52:33 -0800, Claude Henchoz [EMAIL PROTECTED] wrote:

Hi

Is there any way of listing partitions on a (win32) computer without
using WMI?

 Maybe this will work (I skipped A: and B:, but you can include them if
 you want to catch floppy drives with something in them). The 'xxx' is just 
 so as
 not to get a full directory's worth of useless text. If xxx happens to be 
 defined
 it doesn't hurt anything. It's not going to be as fast as using one of the
 win32 api packages to get at GetLogicalDriveStrings, but this works on my 
 NT4:
 (error messages apparently go to stderr, so stdout gets '' which makes the 
 if fail)

  def fakeGetLogicalDriveStrings():
 ... return [c+':' for c in (chr(n) for n in xrange(ord('A'), ord('Z')+1))
 ...   if os.popen('dir %s:\\xxx'%c).read()]
 ...
  fakeGetLogicalDriveStrings()
 ['C:', 'D:', 'E:', 'V:', 'W:']

 Regards,
 Bengt Richter

This will miss any partitions that don't have a drive letter assigned.
It will also give duplicate results for any volumes that have more
than one drive letter.  And it will return an entry for CD or DVD
drives which aren't disk partitions.

Yes, I guess I wasn't paying attention to the focus on partitions.
I wonder if you could try opening the physical disks (there is a weird
path prefix syntax IIRC) and read and interpret MBRs etc. for yourself.
Of course, you would need privilege to open raw disk reading, and you
definitely wouldn't want to make a mistake about read vs write ;-)

If ctypes becomes standard amongst the batteries included, this kind of
stuff should get easier, since I suppose you could use it to get to the
APIs you suggest (DeviceIoControl with IOCTL_DISK_GET_DRIVE_LAYOUT).

Kind of scary the damage you could do as administrator though, given that
on windows people tend to wind up running as administrator by default if they 
own
the machine ;-/

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: instance attributes not inherited?

[Bengt Richter]

On Sun, 15 Jan 2006 18:44:50 -0500, John M. Gabriele [EMAIL PROTECTED] 
wrote:

The following short program fails:
to do what you intended, but it does not fail to do what you programmed ;-)


--- code 
#!/usr/bin/python

class Parent( object ):
 def __init__( self ):
 self.x = 9
 print Inside Parent.__init__()


class Child( Parent ):
 def __init__( self ):
 print Inside Child.__init__()


p1 = Parent()
p2 = Parent()
c1 = Child()
foo = [p1,p2,c1]

for i in foo:
 print x =, i.x
- /code --



yielding the following output:

 output --
Inside Parent.__init__()
Inside Parent.__init__()
Inside Child.__init__()
x = 9
x = 9
x =
Traceback (most recent call last):
   File ./foo.py, line 21, in ?
 print x =, i.x
AttributeError: 'Child' object has no attribute 'x'
 /output -


Why isn't the instance attribute x getting inherited?
It would be if it existed, but your Child defines __init__
and that overrides the Parent __init__, so since you do not
call Parent.__init__ (directly or by way of super), x does not get set,
and there is nothing to inherit.

BTW, inherit is not really what makes x visible as an attribute of the
instance in this case. The x attribute happens to be assigned in 
Parent.__init__,
but that does not make it an inherited attribute of Parent. IOW, Parent.__init__
is a function that becomes bound to the instance and then operates on the 
instance
to set the instance's x attribute, but any function could do that. Or any 
statement
with access to the instance in some way could do it. Inheritance would be if 
there were
e.g. a Parent.x class variable or property. Then Child().x would find that by 
inheritance.

BTW, if you did _not_ define Child.__init__ at all, Child would inherit 
Parent.__init__
and it would be called, and would set the x attribute on the child instance.


My experience with OOP has been with C++ and (more
recently) Java. If I create an instance of a Child object,
I expect it to *be* a Parent object (just as, if I subclass
a Car class to create a VW class, I expect all VW's to *be*
Cars).

That is to say, if there's something a Parent can do, shouldn't
the Child be able to do it too? Consider a similar program:

--- code 
#!/usr/bin/python


class Parent( object ):
 def __init__( self ):
 self.x = 9
 print Inside Parent.__init__()

 def wash_dishes( self ):
 print Just washed, self.x, dishes.


class Child( Parent ):
 def __init__( self ):
 print Inside Child.__init__()


p1 = Parent()
p2 = Parent()
c1 = Child()
foo = [p1,p2,c1]

for i in foo:
 i.wash_dishes()
--- /code ---

But that fails with:

--- output --
Inside Parent.__init__()
Inside Parent.__init__()
Inside Child.__init__()
Just washed 9 dishes.
Just washed 9 dishes.
Just washed
Traceback (most recent call last):
   File ./foo.py, line 24, in ?
 i.wash_dishes()
   File ./foo.py, line 10, in wash_dishes
 print Just washed, self.x, dishes.
AttributeError: 'Child' object has no attribute 'x'
--- /output -

Why isn't this inherited method call working right?
The method call is working fine. It's just that as before
Child.__init__ overrides Parent.__init__ without calling
Parent.__init__ or setting the x attribute itself, so
AttributeError: 'Child' object has no attribute 'x'
is telling the truth. And it's telling that was discovered
at File ./foo.py, line 10, in wash_dishes -- in other
words _inside_ wash_dishes, meaning the call worked, but
you hadn't provided for the initialization of the x attribute
it depended on.

There are various ways to fix this besides calling Parent.__init__
from Child.__init__, depending on desired semantics.

Is this a problem with Python's notion of how OO works?
Nope ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to remove subset from a file efficiently?

[Bengt Richter]

On 13 Jan 2006 23:17:05 -0800, [EMAIL PROTECTED] wrote:


fynali wrote:
 $ cat cleanup_ray.py
 #!/usr/bin/python
 import itertools

 b = set(file('/home/sajid/python/wip/stc/2/CBR333'))

 file('PSP-CBR.dat,ray','w').writelines(itertools.ifilterfalse(b.__contains__,file('/home/sajid/python/wip/stc/2/PSP333')))

 --
 $ time ./cleanup_ray.py

 real0m5.451s
 user0m4.496s
 sys 0m0.428s

 (-: Damn!  That saves a bit more time!  Bravo!

 Thanks to you Raymond.
Have you tried the explicit loop variant with psyco ? My experience is
that psyco is pretty good at optimizing for loop which usually results
in faster code than even built-in map/filter variant.

Though it would just be 1 or 2 sec difference(given what you already
have) so may not be important but could be fun.

OTOH, when you are dealing with large files and near-optimal simple processing 
you are
likely to be comparing i/o-bound processes, meaning differences observed
will be symptoms of os and file system performance more than of the algorithms.

An exception is when a slight variation in algorithm can cause a large change
in i/o performance, such as if it causes physical seek and read patterns of disk
access that the OS/file_system and disk interface hardware can't entirely 
optimize out
with smart buffering etc. Not to mention possible interactions with all the 
other things
an OS may be doing simultaneously switching between things that it accounts 
for as real/user/sys.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Recursive tree list from dictionary

[Bengt Richter]

On Sat, 14 Jan 2006 16:46:29 -0400, David Pratt [EMAIL PROTECTED] wrote:

Hi Allan, Max, and bearophile

Many thanks for your replies to this. The number of levels can be deeper 
than two for creating child, sibling relationships. This can lead to 
futher nesting as shown in my sample result list (the result I am 
attempting to acheive) which is reason that I believe this has to be 
recursive to work. As an illustration, I have altered the source_dict to 
show this by making Geometries the parent of Geometry. I should have 
gone one more level to illustrate this properly - my apologies for not 
doing this better.

source_list = [{'title': 'Project', 'parent':'root'},
   {'title': 'Geometry', 'parent':'Geometries'},
   {'title': 'Soil', 'parent':'root'},
   {'title': 'Geometries', 'parent':'Project'},
   {'title': 'Verticals', 'parent':'Project'},
   {'title': 'Points', 'parent':'Geometry'},
   {'title': 'Layers', 'parent':'Geometry'},
   {'title': 'Water', 'parent':'Geometry'},
   {'title': 'Soiltypes', 'parent':'Soil'}
   ]

Project and Soil in root,
Geometries (Geometries containing Geometry with its siblings) and 
Verticals in Project,
Soiltypes in Soil

In this instance, I am looking for a result list that looks like this:

result_list = [
   # Project
   ['Project', ['Geometries',['Geometry', ['Points', 
'Layers','Water']],'Verticals']],
   # Soil
   ['Soil', ['Soiltypes']]
   ]

I have not yet installed http://sourceforge.net/projects/pynetwork/ but 
will also give this a try. What I need is something like elementree 
minus the xml since I am working with list of dicts obtained from more 
than one type of source.

If you literally want # Project comments in the output, it will have
to be a source code (string) output, e.g., the following generates and
prints both:

 david_pratt.py --
source_list = [{'title': 'Project', 'parent':'root'},
{'title': 'Geometry', 'parent':'Geometries'},
{'title': 'Soil', 'parent':'root'},
{'title': 'Geometries', 'parent':'Project'},
{'title': 'Verticals', 'parent':'Project'},
{'title': 'Points', 'parent':'Geometry'},
{'title': 'Layers', 'parent':'Geometry'},
{'title': 'Water', 'parent':'Geometry'},
{'title': 'Soiltypes', 'parent':'Soil'}
]
children = {}
for d in source_list:
children.setdefault(d['parent'], []).append(d['title'])

def get_progeny(title, d=children):
if title in d: return [title] + [get_progeny(child) for child in d[title]]
else: return title

source = ['result_list = [']
result_list = []
for child in children['root']:
source.append('# %s'%child)
source.append('%r,'% get_progeny(child))
result_list.append(get_progeny(child))
source.append(']')
result_list_source = '\n'.join(source)
print result_list_source
print result_list


[18:20] C:\pywk\clppy24 david_pratt.py
result_list = [
# Project
['Project', ['Geometries', ['Geometry', 'Points', 'Layers', 'Water']], 
'Verticals'],
# Soil
['Soil', 'Soiltypes'],
]
[['Project', ['Geometries', ['Geometry', 'Points', 'Layers', 'Water']], 
'Verticals'], ['Soil', '
Soiltypes']]

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: On Numbers

[Bengt Richter]

On Sat, 14 Jan 2006 18:18:12 -0500, Mike Meyer [EMAIL PROTECTED] wrote:

In the discussion of equality, the issue that decimal('3.0') == 3.0 is
False came up as a reason for changing the behavior of ==. The problem
with this is that the proposed change doesn't really fix anything, it
just gives different wrong behavior. The correct fix would seem to be
fixing python's handling of numbers.

It's not clear from the PEP why test for equality always fails. While
the PEP says that interactions with floats are disallowed as to
tricky, comparisons seem to be an exception. In particular, equality
comparisons seem to always return false, whereas checking for
magnitude seems to work.

What appears to be missing in Python is a coherent model for all
numberic types. Other languages have this, so there's no obvious
reason that Python can't. Or maybe Python does, and I've overlooked
it. In that case, pointers to documentation would be appreciated.
If it doesn't exist, then I think this is something that should be
addressed for Py3K.

I'd like to work on that. The idea would be that all the numeric types
are representations of reals with different properties that make them
appropriate for different uses. The goal would be a model for a
numeric type that captures those properties in a way that allows them
to be combined to give sane behavior without being to tricky.

To get started on that, I want to consider how other languages with
rich numeric type systems model them. I'm going to look at a couple of
languages in the LISP family. I'd like to solicit the community for
the names of languages (and pointers to references, if possible!) with
a coherent view of a rich set of numeric types.

I suppose you have already found scheme's rsr5 spec, e.g.,
http://www.schemers.org/Documents/Standards/R5RS/

while thinking about numbers, the issue of equivalence that got us here
is also explored in scheme

http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-9.html#%_sec_6.1
and the section that follows in  on numbers

http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-9.html#%_sec_6.2

for common lisP
http://www.lispworks.com/documentation/HyperSpec/Front/index.htm
you can also chase lots of interesting links (some dead though) from
http://www.cetus-links.org/oo_clos.html
cltl2 is at
http://www.cs.cmu.edu/Groups/AI/html/cltl/cltl2.html

Now try to get some work done ;-)

BTW, IIRC Ada's numbers include some interesting provisions for representing 
numbers
in fixed point with specified fractional accuracy
I google and found
http://www.adaic.org/standards/95lrm/html/RM-0-1.html
browsing a bit, scalar types are at
http://www.adaic.org/standards/95lrm/html/RM-3-5.html
integers include unsigned by way of genera modular definition
http://www.adaic.org/standards/95lrm/html/RM-3-5-4.html
and here is the fixed point
http://www.adaic.org/standards/95lrm/html/RM-3-5-9.html
Whoo, I haven't been into that stuff almost since Ada was green ;-)
Makes me wonder if generators could usefully have rendezvous('?, es?) ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: flatten a level one list

[Bengt Richter]

On Fri, 13 Jan 2006 07:48:39 -0800, [EMAIL PROTECTED] (Alex Martelli) wrote:

Nick Craig-Wood [EMAIL PROTECTED] wrote:
   ...
 I agree with Guido about the special case, but I disagree about the
 error message.  Not being able to use sum(L,) reduces the
 orthogonality of sum for no good reason I could see.

I agree.

Having sum(L,'') work but be O(N squared) would be an attractive
Are you saying ''.join is O(N squared)? Or that sum would not be
allowed to used the same/equivalent implementation algorithms if it were allowed
to work on strings?

nuisance within the meaning of the law; it's bad enough that sum(L,[])
etc are that way, but at least beginners want to concatenate lists (or
tuples, arrays, etc) much less often than they want to concatenate
strings.  sum is meant to work on _numbers_ -- the reason it takes that
If sum is meant to work on _numbers_, why not call it numsum, to suggest
the restriction? ISTM the usual assumption in python is non-restrictive duck 
typing,
and the expectation would be no more restriction than in reduce(operator.add, 
L, init_val).

second optional parameter is essentially to be able to specify the
``type'' of numbers.  In retrospect it might have been better to have a
single argument (or interpret multiple ones like min or max do, maybe),
forcing the starting value to be integer 0.

So it was an accident that user types are permitted?

  class C:
 ... def __getattr__(self, attr): print attr;raise AttributeError
 ...
  sum([C(),C()])
 __coerce__
 __radd__
 Traceback (most recent call last):
   File stdin, line 1, in ?
 TypeError: unsupported operand type(s) for +: 'int' and 'instance'
  sum([C(),C()],C())
 __coerce__
 __add__
 __coerce__
 __radd__
 Traceback (most recent call last):
   File stdin, line 1, in ?
 TypeError: unsupported operand type(s) for +: 'instance' and 'instance'

Yet user subclassing of str does not yield an acceptable user type?

  S = type('S',(str,),{})
  sum(S(i) for i in xrange(5))
 Traceback (most recent call last):
   File stdin, line 1, in ?
 TypeError: unsupported operand type(s) for +: 'int' and 'S'
  sum(S(i) for i in xrange(5), S('start:'))
   File stdin, line 1
 SyntaxError: invalid syntax
  sum((S(i) for i in xrange(5)), S('start:'))
 Traceback (most recent call last):
   File stdin, line 1, in ?
 TypeError: sum() can't sum strings [use ''.join(seq) instead]

Although, notice

  class SI(str):
 ... def __add__(self, other): return SI(str.__add__(self, str(other)))
 ... __radd__ = __add__
 ...
  sum((SI(i) for i in xrange(5)))
 '001234'

But:
  sum((SI(i) for i in xrange(5)), SI('start:'))
 Traceback (most recent call last):
   File stdin, line 1, in ?
 TypeError: sum() can't sum strings [use ''.join(seq) instead]
vs
  reduce(operator.__add__, (SI(i) for i in xrange(5)), SI('start:'))
 'start:01234'


Which seems to boil down to incomplete restrictions on duck typing.
Maybe Guido will want to complete it, but ISTM your original implementation
delegating string addition implementation to ''.join was reasonable.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Failing unittest Test cases

[Bengt Richter]

On Tue, 10 Jan 2006 11:13:20 +0100, Peter Otten [EMAIL PROTECTED] wrote:

Duncan Booth wrote:

 Peter Otten wrote:
 
 Marking a unittest as should fail in the test suite seems just wrong
 to me, whatever the implementation details may be. If at all, I would
 apply a I know these tests to fail, don't bother me with the messages
 for now filter further down the chain, in the TestRunner maybe.
 Perhaps the code for platform-specific failures could be generalized?
 
 It isn't marking the test as should fail it is marking it as should
 pass, but currently doesn't which is a very different thing.

You're right of course. I still think the currently doesn't pass marker
doesn't belong into the test source.

Perhaps in a config file that can specify special conditions re running
identified tests? E.g., don't run vs run and report (warn/fail/info)
changed result (e.g. from cached result) vs run and report if pass etc.

Then if code change unexpectedly makes a test work, the config file can just
be updated, not the test.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I make a dictionary that marks itself when it's modified?

[Bengt Richter]

On Thu, 12 Jan 2006 18:28:54 +, Steve Holden [EMAIL PROTECTED] wrote:

[EMAIL PROTECTED] wrote:
 [EMAIL PROTECTED] wrote:
 
It's important that I can read the contents of the dict without
flagging it as modified, but I want it to set the flag the moment I add
a new element or alter an existing one (the values in the dict are
mutable), this is what makes it difficult. Because the values are
mutable I don't think you can tell the difference between a read and a
write without making some sort of wrapper around them.

Still, I'd love to hear how you guys would do it.
 
 
 if the dictionary is small and speed not important, you can wrap it 
 in a class catching __getitem__ and __setitem__ and testing
 if repr(self) changes.
 
 
d = {1: [a, b],
  2: [b, c]}

d[1][0] = 3

How would this work? __getitem__() will be called once, to get a 
reference to the list, and so there's no opportunity to compare the 
'before' and 'after' repr() values.

You are right, but OTOH the OP speaks of a flagging the dict as modified.
If she made e.g., modified a property of the dict subclass, then
retrieving the the modified flag could dynamically check current state repr
vs some prior state repr. Then the question becomes modified w.r.t. what prior 
state?

This lets the OP ask at any time whether the dict is/has_been modified, but 
it's not a basis
for e.g., a modification-event callback registry or such.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: More than you ever wanted to know about objects [was: Is everything a refrence or isn't it]

[Bengt Richter]

On Fri, 13 Jan 2006 03:12:00 +, Steve Holden [EMAIL PROTECTED] wrote:
[...]
Note to nitpickers
--
Please note that I *am* oversimplifying here, and the nitpickers will 
undoubtedly find many threadsworth of valuable material here. The point 
is to develop an understanding of the *principles*. Once someone has 
that they can pick up the details as they need them, which 98% of Python 
users don't feel the need to. I don't even claim to know the whole 
story, but I *do* know enough to explain it in principle. So by all 
means correct me where I am demonstrably wring, but *please* don't just 
  ^-- spelling is wring ;-)
add complexity that doesn't aid understanding.
[...]
When you write

   foo = instance.method(arg1, arg2)

you are almost always calling a function defined inside the instance 
type's class definition (though this being Python there's absolutely 
nothing to stop instances having callable attributes of their own too: 
we are discussing beginners' Python here).

Under the hood the interpreter looks for an attribute called method in 
the instance. Failing to find it, it then looks in the instance's type.
UIAM,  for new-style classes this is wrongly putting the instance attribute
lookup before the mro logic. For new-style, the interpreter goes down the
type(instance).mro() chain looking for the attribute called method
until it finds a method attribute whose value has __get__ method AND a 
__set__ method
(which makes it a data descriptor, and not overridable by an instance 
attribute). If
found, it will call the method.__get__ method with the instance as an argument. 
The __get__ method
then returns (normally) the bound method. If it finds method without __get__ 
and __set__ methods,
it will ignore it, except that it may revisit it if after the whole mro chain 
fails, and there isn't
any method attribute on the instance either. In that case, it will look for 
method again along the
mro, and if it has a __get__ method, it will form a bound method (this is the 
typical method attribute
in the form of a function, which has a __get__ method), or if there is no 
__get__ method, it will
return the value as ordinary class variable. The re-scan of mro is not literal, 
hopefully. I'm just
trying to express the logic. Also special shortcuts apply for methods of 
builtins, I believe.

Old-style classes live alongside new ones -- I suspect -- by virtue of 
type(someinstance) being
type 'instance' and type(someinstance).__getattribute__ thus implementing 
old-style stuff instead
of the type(someinstance).__getattribute__ of new-style instances, which 
inherit their __getattribute__
either from object.__getattribute__ or type.__getattribute__, or a user 
definition.


Of course it can fail to find it there, too. If the type is defined as a 
specialisation of some other type (a subclass of the other type - 
type2 is like type1 but with the following differences) then the 
interpreter will consult the other type, and so on and so on. I am 
deliberately ignoring multiple inheritance (typeC is like typeB, and 
it's also like typeA) here, but the essentials are the same. We say the 
subclass inherits the attributes of the superclass.

If the interpreter keeps walking up the inheritance tree in this way 
without finding the attribute it's looking for, it will eventually 
arrive at the type object, because ultimately every object in Python 
is defined as a subclass of (a subclass of (a subclass of ...)) object.
Unless the attribute access started on a type, in which case it will check
for type.method before object.method, UIAM. BTW, in order to find either
kind of attribute, there will first be a search for 
type(instance).__getattribute__,
which is found as object.__getattribute__ or type.__getattribute__ unless 
overridden
(or instance is of an old-style class as mentioned above).
I suspect these then implement the check for __getattr__, which can be user 
defined
to intercept attribute access on particular objects in the search chain, after
higher-priority stuff fails.

[... Can't ... keep .. eyes ... open ... for ... rest ...]
(not the fault of your prose, had a beer ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: can't solve an exercise-help me with it

[Bengt Richter]

On Thu, 12 Jan 2006 05:51:10 +0100, hossam [EMAIL PROTECTED] wrote:

I'm studying python newly and have an exercise that is difficult for me as a 
beginner.Here it is :
Write a program that approximates the value of pi by summing the terms of 
this series:
4/1-4/3+4/5-4/7+4/9-4/11+ The program should prompt the user for n, the 
number of terms to sum and then output the sum of the first n terms of this 
series.

any help would be appreciated.

Is this homework?
What have you tried so far?
Hints: raw_input, int. And sum, if you want a one-liner.
Hint2: try 4/7 and 4.0/7 interactively

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do you have real-world use cases for map's None fill-in feature?

[Bengt Richter]

On 10 Jan 2006 00:47:36 -0800, Raymond Hettinger [EMAIL PROTECTED] wrote:

[Bengt Richter]
 What about some semantics like my izip2 in
 
 http://groups.google.com/group/comp.lang.python/msg/3e9eb63a1ddb1f46?hl=en

 (which doesn't even need a separate name, since it would be backwards 
 compatible)

 Also, what about factoring sequence-related stuff into being methods or 
 attributes
 of iter instances? And letting iter take multiple sequences or 
 callable/sentinel pairs,
 which could be a substitute for izip and then some? Methods could be called 
 via a returned
 iterator before or after the first .next() call, to control various 
 features, such as
 sentinel testing by 'is' instead of '==' for callable/sentinel pairs, or 
 buffering n
 steps of lookahead supported by a .peek(n) method defaulting to .peek(1), 
 etc. etc.
 The point being to have a place to implement universal sequence stuff.

ISTM, these cures are worse than the disease ;-)
Are you reacting to my turgidly rambling post, or to

  from ut.izip2 import izip2 as izip
  it = izip('abc','12','ABCD')
  for t in it: print t
 ...
 ('a', '1', 'A')
 ('b', '2', 'B')

Then after a backwards-compatible izip, if the iterator has
been bound, it can be used to continue, with sentinel sustitution:

  for t in it.rest('sentinel'): print t
 ...
 ('c', 'sentinel', 'C')
 ('sentinel', 'sentinel', 'D')

or optionally in sentinel substitution mode from the start:

  for t in izip('abc','12','ABCD').rest('sentinel'): print t
 ...
 ('a', '1', 'A')
 ('b', '2', 'B')
 ('c', 'sentinel', 'C')
 ('sentinel', 'sentinel', 'D')

Usage-wise, this seems not too diseased to me, so I guess I want to make sure
this is what you were reacting to ;-)

(Implementation was just to hack together a working demo. I'm sure it can be 
improved upon ;-)



 Even if there is little use for continuing in correct code, IWT getting
 at the state of the iterator in an erroroneous situation would be a benefit.
 Being able to see the result of the last attempt at gathering tuple elements
 could help. (I can see reasons for wanting variations of trying all streams
 vs shortcutting on the first to exhaust though).

On the one hand, that seems reasonable.  On the other hand, I can't see
how to use it without snarling the surrounding code in which case it is
probably better to explicitly manage individual iterators within a
while loop.

The above would seem to allow separation of concerns. I.e., if you care why
a normal iteration terminated, you can test after the fact. I.e., if all 
sequences
were the same length, the .rest() iterator will be empty. And if you don't care 
at
all about possible data, you can just try: it.rest().next() and catch 
StopIteration
to check.

BTW, is there any rule against passing information with StopIteration?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Augmented generators?

[Bengt Richter]

On Tue, 10 Jan 2006 18:37:13 GMT, Andrew Koenig [EMAIL PROTECTED] wrote:

Can anyone think of an easy technique for creating an object that acts like 
a generator but has additional methods?

For example, it might be nice to be able to iterate through an associative 
container without having to index it for each element.  Right now, I can say

i = iter(d)

and then repeatedly calling i.next() gives us the keys for the elements. 
But to get the corresponding value requires us to look up the key.

Of course one could define a generator that yields key-value pairs, along 
the following lines:

def kviter(d):
for i in d:
yield i, d[i]

to hide the lookup.  But this yields a tuple even when you don't want it. 
In other words, I must now write

for k, v in kviter(d):
# whatever

and I can think of situations in which I don't really want both the key and 
the value all the time.

So what I really want is something like this:

it = augiter(d)
for i in it:
if some condition on i:
foo(it.value())

In other words, I want it to support both the next and value methods (or 
next and something else)

Of course I can write such a beast as a class, but that prevents me from 
taking advantage of the yield statement in its implementation.

It does? (this just occurred to me, so there may be some gotcha ;-)

  class augiter(object):
 ... def __init__(self, d): self.d = d
 ... def __iter__(self):
 ... for k in self.d: self.value = self.d[k]; yield k
 ...
  it = augiter(dict(enumerate('abcd')))
  for i in it:
 ... if i%2: print i, it.value
 ...
 1 b
 3 d

You could of course store self._value and def value(self):return self._value to 
be closer to your syntax)

So my question is: Can you think of an easy way to write something that 
looks like a generator (using yield), but can also incorporate methods other 
than next?

See above.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Augmented generators?

[Bengt Richter]

Sorry to reply this way. I saw this on google, but neither this nor my previous 
post
has shown up yet on my news server. Wonder what all this delay is lately ;-/

From: Paul Rubin http://[EMAIL PROTECTED]
Subject: Re: Augmented generators?
Date: 10 Jan 2006 11:03:39 -0800
Message-ID: [EMAIL PROTECTED]

Andrew Koenig [EMAIL PROTECTED] writes:
 Can anyone think of an easy technique for creating an object that acts like 
 a generator but has additional methods?

 For example, it might be nice to be able to iterate through an associative 
 container without having to index it for each element.  

Normally you'd define a class and give it __iter__ and next operations.
I guess that can get messy.  Does it qualify as easy for your purposes?

 Of course I can write such a beast as a class, but that prevents me from 
 taking advantage of the yield statement in its implementation.

You can make an internal function that's a generator with a yield
statement (or a generator expression instead of a function, if simple
enough).  The class's 'next' method would loop through the generator
and return each value from it.

Let me try your example:

This suffers from the same problem as my first go (an exhausted iterator
is not supposed to restart).

  class kviter:
 ...def __init__(self, d):
 ...   self.d = d
 ...def __iter__(self):
 ...   def iter1(d, s=self):
 ...  for k in d:
 ... # lambda not really needed here, but you wanted value to
 ... # be callable instead of just an attribute
 ... s.value = (lambda r=d[k]: r)
 ... yield k
 ...   return iter1(self.d)
 ...

Using my example with your iterator:

  it = kviter(dict(enumerate('abcd')))
  for i in it:
 ... if i%2: print i, it.value()
 ...
 1 b
 3 d

Ok, but this shouldn't happen at this point:

  for i in it:
 ... if i%2: print i, it.value()
 ...
 1 b
 3 d

Whereas,

  class augiter(object):
 ... def __init__(self, d):
 ... self._it = self._gen(d)
 ... def __iter__(self): return self
 ... def _gen(self, d):
 ... for k, self._v in d.items(): yield k
 ... def next(self): return self._it.next()
 ... def value(self): return self._v
 ...
  it = augiter(dict(enumerate('abcd')))
  for i in it:
 ... if i%2: print i, it.value()
 ...
 1 b
 3 d
  for i in it:
 ... if i%2: print i, it.value()
 ...
 

Letting __init__ create the generator instance by calling a bound
method coded with yield makes integrating the latter style pretty easy,
even though you still need __iter__ and next methods.

Hm, you could factor that out into a base class though:
(then you just need the convention that a derived class must define at least
the _gen method. Then augment to taste)

  class AugiterBase(object):
 ... def __init__(self, *args, **kw):
 ... self._it = self._gen(*args, **kw)
 ... def __iter__(self): return self
 ... def next(self): return self._it.next()
 ...
  class DervAugit(AugiterBase):
 ... def _gen(self, d):
 ... for k, self._v in d.items(): yield k
 ... def value(self): return self._v
 ...
  it = DervAugit(dict(enumerate('abcd')))
  for i in it:
 ... if i%2: print i, it.value()
 ...
 1 b
 3 d
  for i in it:
 ... print i, it.value()
 ...
  it = DervAugit(dict(enumerate('abcd')))
  for i in it:
 ... print i, it.value()
 ...
 0 a
 1 b
 2 c
 3 d

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: itertools.izip brokeness

[Bengt Richter]

On 9 Jan 2006 08:19:21 GMT, Antoon Pardon [EMAIL PROTECTED] wrote:

Op 2006-01-05, Bengt Richter schreef [EMAIL PROTECTED]:
 On 5 Jan 2006 15:48:26 GMT, Antoon Pardon [EMAIL PROTECTED] wrote:
[...]
 But you can fix that (only test is what you see ;-) :

Maybe, but not with this version.

  from itertools import repeat, chain, izip
  it = iter(lambda z=izip(chain([3,5,8],repeat(Bye)), 
  chain([11,22],repeat(Bye))):z.next(), (Bye,Bye))
  for t in it: print t
  ...
  (3, 11)
  (5, 22)
  (8, 'Bye')

 (Feel free to generalize ;-)

The problem with this version is that it will stop if for some reason
each iterable contains a 'Bye' at the same place. Now this may seem
far fetched at first. But consider that if data is collected from
ISTM the job of choosing an appropriate sentinel involves making
that not only far fetched but well-nigh impossible ;-)

experiments certain values may be missing. This can be indicated
by a special Missing Data value in an iterable. But this Missing
Data value would also be the prime canidate for a fill parameter
when an iterable is exhausted.

ISTM that confuses missing data with end of data stream.
I assumed your choice of terminating sentinel (Bye) would not have
that problem ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do you have real-world use cases for map's None fill-in feature?

[Bengt Richter]

On 7 Jan 2006 23:19:41 -0800, Raymond Hettinger [EMAIL PROTECTED] wrote:

I am evaluating a request for an alternate version of itertools.izip()
that has a None fill-in feature like the built-in map function:

 map(None, 'abc', '12345')   # demonstrate map's None fill-in feature
[('a', '1'), ('b', '2'), ('c', '3'), (None, '4'), (None, '5')]

I don't like not being able to supply my own sentinel. None is too common
a value. Hm, ... bf warning unless maybe it can also be a type that we can 
instantiate with
really-mean-it context level like None(5) ;-)

  map(None(5), 'abc', '12345')   # demonstrate map's None fill-in feature
 [('a', '1'), ('b', '2'), ('c', '3'), (None(5), '4'), (None(5), '5')]

But seriously, standard sentinels for missing data and end of data might be 
nice to have,
and to have produced in appropriate standard contexts. Singleton string subclass
instances NOD and EOD? Doesn't fit with EOF=='' though.
/bf warning

The movitation is to provide a means for looping over all data elements
when the input lengths are unequal.  The question of the day is whether
that is both a common need and a good approach to real-world problems.
The answer to the question can likely be found in results from other
programming languages or from real-world Python code that has used
map's None fill-in feature.

What about some semantics like my izip2 in
http://groups.google.com/group/comp.lang.python/msg/3e9eb63a1ddb1f46?hl=en

(which doesn't even need a separate name, since it would be backwards 
compatible)

Also, what about factoring sequence-related stuff into being methods or 
attributes
of iter instances? And letting iter take multiple sequences or 
callable/sentinel pairs,
which could be a substitute for izip and then some? Methods could be called via 
a returned
iterator before or after the first .next() call, to control various features, 
such as
sentinel testing by 'is' instead of '==' for callable/sentinel pairs, or 
buffering n
steps of lookahead supported by a .peek(n) method defaulting to .peek(1), etc. 
etc.
The point being to have a place to implement universal sequence stuff.

I scanned the docs for Haskell, SML, and Perl and found that the norm
for map() and zip() is to truncate to the shortest input or raise an
exception for unequal input lengths.  I scanned the standard library
and found no instances where map's fill-in feature was used.  Likewise,
I found no examples in all of the code I've ever written.

The history of Python's current zip() function serves as another
indicator that the proposal is weak.  PEP 201 contemplated and rejected
the idea as one that likely had unintended consequences.  In the years
since zip() was introduced in Py2.0, SourceForge has shown no requests
for a fill-in version of zip().

My request for readers of comp.lang.python is to search your own code
to see if map's None fill-in feature was ever used in real-world code
(not toy examples).  I'm curious about the context, how it was used,
and what alternatives were rejected (i.e. did the fill-in feature
improve the code).

Also, I'm curious as to whether someone has seen a zip fill-in feature
employed to good effect in some other programming language, perhaps
LISP or somesuch?
ISTM in general there is a chicken-egg problem where workarounds are easy.
I.e., the real question is how many workaround situations there are
that would have been handled conveniently with a builtin feature,
and _then_ to see whether the convenience would be worth enough.

Maybe a few real-word code examples and experiences from other
languages will shed light on the question of whether lock-step
iteration has meaning beyond the length of the shortest matching
elements.  If ordinal position were considered as a record key, then
the proposal equates to a database-style outer join operation (where
data elements with unmatched keys are included) and order is
significant.  Does an outer-join have anything to do with lock-step
iteration?  Is this a fundamental looping construct or just a
theoretical wish-list item?  IOW, does Python really need
itertools.izip_longest() or would that just become a distracting piece
of cruft?
Even if there is little use for continuing in correct code, IWT getting
at the state of the iterator in an erroroneous situation would be a benefit.
Being able to see the result of the last attempt at gathering tuple elements
could help. (I can see reasons for wanting variations of trying all streams
vs shortcutting on the first to exhaust though).

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is 'everything' a refrence or isn't it?

[Bengt Richter]

On Mon, 09 Jan 2006 07:40:18 -, Donn Cave [EMAIL PROTECTED] wrote:

Quoth [EMAIL PROTECTED]:
| Fredrik Lundh wrote:
| ...snip...
|  afaik, the Python Language Reference never defines the word reference.
|  It carefully defines words like object and value, though, and terms 
like
|  call by object or call by object reference are perfectly understandable
|  if you use the words as they are defined in the language reference.
|
| It (sec. 3.1, Objects, values and types) is not what I would
| call a good definition .  About values it says only
|
| - that they are something that all objects have.
| - they can be mutable or immutable.
|
| It then has a few sentences about mutability, so after reading
| it you will know that, whatever a value is, it can be changed
| for some objects but not others.  But what exactly it is that is
| being changed is still a mystery.
| Further down it talks about container objects containing
| references to other objects and of those being part of it's
| value.  Part?  What does it mean to contain?  Can one
| determine from reading this, if an attribute value is part
| of the object's value?  (I couldn't).
|
| On my list on Python Doc problems that need fixing, is
| defintion of object values and it has been on there
| for nearly a year.

So you've had time to think about how you would define value, in a
few words.  Any ideas?

I find the topic difficult, myself.  I think you really have to apply
some context to the question, so there may not be any satisfactory
definition for the language reference.

But maybe it would be simple with the right focus.  If we could somehow
define value, how would that help?  I mean, presumably we need to
understand all this stuff because we want to write some software, and
if we dive in without understanding, our attempts will be plagued with
conceptual errors.  Is there something about value in particular that
seems to be a problem here?  ``No, you idiot, that's not a value -
THIS is a value!''

IMO the problem is that our intuitive notion of value is ususally really of
an abstract value, like an integer value or a real or maybe a member of a finite
abstract set. But in programming we are always dealing with various concrete
representations of values, not values themselves. To get to a value, you have
to interpret the representation. To perform abstract operations, you have to
manipulate concrete representations, and guarantee that the resulting 
representations
really do represent the abstract results of the abstract operation.

People are so used to interpreting representations that they do it 
unconsciously.
Mostly the metonymic use of representations in discussing abstractions is a
taken-for-granted short cut for avoiding tiresome pedantry, but sometimes, I 
believe,
it lies at the heart of difficulties such as in trying to define value.

IMO the first step in defining value should be to declare that it always 
refers
to some abstract value, which is always an interpretation of some concrete 
representation.

I suspect that describing things in terms of abstract graphs and their 
transformations
would be useful. E.g., starting with python's objects (in the abstract) as 
nodes,
and references as directed arcs.

More on this later, perhaps, but I have to go now ...

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Viewing Binary Data

[Bengt Richter]

On 6 Jan 2006 09:15:50 -0800, Cuyler [EMAIL PROTECTED] wrote:

Hello,

I would like to display a file in its binary form (1s and 0s), but I'm
having no luck... Any thoughts would be most appreciated.

What have you tried? having no luck doesn't tell us much, although it is
a phrase previously seen in use ;-/
Is this homework?

If the first character were 'C' what order would you like to see the bits?
(e.e., 111 (big-endian l-r) vs 111 (little-endian l-r)
Do you want to convert the whole file like a hex dump, except binary?
Want to give a clue as to output format? Gapped? Ascii at the right?
Bit numbers or hex byte offsets at the left?
Do you need a one-liner solution?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: question about mutex.py

[Bengt Richter]

On Sun, 08 Jan 2006 10:24:43 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

[EMAIL PROTECTED] wrote:
 yes, I read it, and I even know about threading's existence. I just
 thought that if something claims to be atomic, it better should be.

I think the term atomic is meaningful only when the context is known. 
  For example, atomic operations in the Python interpreter are 
certainly not atomic within the larger context of the CPU, and atomic 
CPU operations are not necessarily atomic in the context of a system 
with multiple CPUs.  If the context for mutex.py explicitly excludes 
multi-threading then you have to interpret atomic in whatever context 
that defines.

Atomic means trademarked by a company that used to use that name in the 50's
to describe and identify a line toys it put in its breakfast cereal boxes.
The rights are now owned by an IP scavenging company which is trying to sell
them for stock in another IP scavenger with more money left, so be careful.
There's also some related talk of a patent on a method of using semiotic 
elements
in association with the distribution of digital generic products to make them
distinguishable for any business purpose whatever. IP principles established 
with
corn flakes and decoder rings are thought to translate perfectly to the digital
domain of FOSS distros including anything toy-like.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiway Branching

[Bengt Richter]

On 8 Jan 2006 18:59:28 GMT, [EMAIL PROTECTED] wrote:

  I need to look at two-byte pairs coming from a machine, and interpret the
meaning based on the relative values of the two bytes. In C I'd use a switch
statement. Python doesn't have such a branching statement. I have 21
comparisons to make, and that many if/elif/else statements is clunky and
inefficient. Since these data are coming from an OMR scanner at 9600 bps (or
faster if I can reset it programmatically to 38K over the serial cable), I
want a fast algorithm.

  The data are of the form:

  if byte1 == 32 and byte2 == 32:
   row_value = 0
  elif byte1 == 36 and byte2 == 32:
   row_value = natural
   ...
  elif byte1 == 32 and byte2 == 1:
   row_value = 5
  elif byte1 == 66 and byte2 == 32:
   row_value = 0.167

  There are two rows where the marked response equates to a string and 28
rows where the marked response equates to an integer (1-9) or float of
defined values.

  Suggestions appreciated.

Set up a dict to map your byte pairs to values, e.g.,

  pairvalues = dict([
 ... ((32,32), 0),
 ... ((36,32), natural),
 ... # ...
 ... ((32, 1), 5),
 ... ((66,32), 0.167)
 ... ])

Then you can access the values like:

  row_value = pairvalues.get((36,32), 'default')
  row_value
 'natural'

The .get method allows you to specify a default, in case you get an unexpected 
pair.
You could also use pairvalues[(byte1,byte2)] and catch the KeyError exception 
for
unexpected pairs.

  for byte1, byte2 in (66,32), (32,1), (36,32), (32,32), (20,20):
 ... print '%10s = %r' %((byte1,byte2), pairvalues.get((byte1,byte2), 
'DEFAULT ??'))
 ...
   (66, 32) = 0.16701
(32, 1) = 5
   (36, 32) = 'natural'
   (32, 32) = 0
   (20, 20) = 'DEFAULT ??'

HTH

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiway Branching

[Bengt Richter]

On Sun, 8 Jan 2006 20:31:49 +0100, Fredrik Lundh [EMAIL PROTECTED] wrote:

[EMAIL PROTECTED] wrote:

   I need to look at two-byte pairs coming from a machine, and interpret the
 meaning based on the relative values of the two bytes. In C I'd use a switch
 statement. Python doesn't have such a branching statement. I have 21
 comparisons to make, and that many if/elif/else statements is clunky and
 inefficient. Since these data are coming from an OMR scanner at 9600 bps (or
 faster if I can reset it programmatically to 38K over the serial cable), I
 want a fast algorithm.

   The data are of the form:

   if byte1 == 32 and byte2 == 32:
 row_value = 0
   elif byte1 == 36 and byte2 == 32:
 row_value = natural
...
   elif byte1 == 32 and byte2 == 1:
 row_value = 5
   elif byte1 == 66 and byte2 == 32:
 row_value = 0.167

   There are two rows where the marked response equates to a string and 28
 rows where the marked response equates to an integer (1-9) or float of
 defined values.

DATA_MAP = {
chr(32)+chr(32): 0,
chr(36)+chr(32): natural,
...
chr(32)+chr(1): 5,
chr(66)+chr(32): 0.167,
}

...

row_value = DATA_MAP[source.read(2)]

# or: row_value = DATA_MAP.get(source.read(2), DEFAULT)

Much better than my version, since you went beyond the OP's code to what he said
he was trying to do (look at two-byte pairs coming from a machine ... over the 
serial cable).
I just went for a direct translation of the code, which is nearly always a 
mistake.
I should know better. I guess I do, since I always rant about requirements ;-/
Also don't know why I chose to use dict([]) vs {}, since there's no bare key 
names to
clean the quotes off of. Oh well.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: decorator question

[Bengt Richter]

On Sun, 08 Jan 2006 23:26:28 +0100, =?ISO-8859-1?Q?Sch=FCle_Daniel?= [EMAIL 
PROTECTED] wrote:
[...]

the code above works fine
but I am wondering wheather it's possible to
write something like this

  def timelogger(f, logfile=sys.stdout):
... def wrapper(*a,**kw):
... logfile.write(started at %s % time.ctime())
... t0 = time.time()
... f(*a, **kw)
... t1 = time.time()
... logfile.write(ended at %s % time.ctime())
... logfile.write(diff = %f %s % (t1-t0, sec))
... return wrapper

  import time
  @timelogger(file(hierher, a)) ###  (1)
... def loops(a,b,c):
... sum = 0
... for i in range(a):
... for j in range(b):
... for k in range(c):
... sum += 1
...

(1) fails to compile
is it possible to pass parameters to a decorator function?

Yes, but then the function must return the same kind of thing
a bare decorator-function name would have, which is a function
able to take a single argument of a function and return a function.

So your decorator won't take f as an argument, just the optional logfile,
and it will return a function that does the wrapping like the original 
decorator.

  import sys, time
  def timelogger(logfile=sys.stdout):
 ...def deco(f):
 ...def wrapper(*a,**kw):
 ...logfile.write(started at %s\n % time.ctime())
 ...t0 = time.time()
 ...f(*a, **kw)
 ...t1 = time.time()
 ...logfile.write(ended at %s\n % time.ctime())
 ...logfile.write(diff = %f %s\n % (t1-t0, sec))
 ...return wrapper
 ...return deco
 ...
  @timelogger() # stdout
 ... def foo(): pass
 ...
  foo()
 started at Sun Jan 08 14:13:55 2006
 ended at Sun Jan 08 14:13:55 2006
 diff = 0.00 sec
  foo()
 started at Sun Jan 08 14:14:02 2006
 ended at Sun Jan 08 14:14:02 2006
 diff = 0.00 sec
  @timelogger() # stdout
 ... def foo(dt): time.sleep(dt)
 ...
  foo(5)
 started at Sun Jan 08 14:14:59 2006
 ended at Sun Jan 08 14:15:04 2006
 diff = 5.007000 sec
  foo(.5)
 started at Sun Jan 08 14:15:16 2006
 ended at Sun Jan 08 14:15:17 2006
 diff = 0.501000 sec
 
 
Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What is the slickest way to transpose a square list of lists (tuple of tuples)?

[Bengt Richter]

On Sun, 08 Jan 2006 15:21:59 -0600, Brian van den Broek [EMAIL PROTECTED] 
wrote:

Gerard Brunick said unto the world upon 08/01/06 01:27 PM:
 My way is ugly.  These has to be a better way.
 
 Thanks,
 Gerard

If you'd posted your way, I might well have seen if I could do it in a 
nicer fashion. But, since for all I know, my best efforts would result 
in the approach you already have, I'm unlikely to put the effort in.

I suspect I'm not alone. You might do well to show the approach you 
are unhappy with.

OTOH, he's probably just trying to get someone to remember zip-abuse for him.
Hope it's not homework ;-)

  sq = [[r+c for c in 'abc'] for r in '123']
  for r in sq: print r
 ...
 ['1a', '1b', '1c']
 ['2a', '2b', '2c']
 ['3a', '3b', '3c']
  for r in (zip(*sq)): print r
 ...
 ('1a', '2a', '3a')
 ('1b', '2b', '3b')
 ('1c', '2c', '3c')


Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: is there any lib can split string in this way?

[Bengt Richter]

On Mon, 9 Jan 2006 13:59:44 +0800, Leo Jay [EMAIL PROTECTED] wrote:

I want to split a string like this:
'abc  def  this is a test  ok'
into:
['abc', 'def', 'this is a test', 'ok']

is there any lib meet my need?

Narrowly interpreting your requirements (only quotes are with
double quotes (never containing escaped same) and strip quotes off)
and tested only as you see ;-)

  import re
  rx = re.compile(r'([^]*)|(\w+)')
  s = 'abc  def  this is a test  ok'
  [a or b for a,b in rx.findall(s)]
 ['abc', 'def', 'this is a test', 'ok']

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Does Python allow access to some of the implementation details?

[Bengt Richter]

On Sat, 07 Jan 2006 14:05:18 +0100, Claudio Grondi [EMAIL PROTECTED] wrote:
[...]
What I am also looking for is a conversion to base 256 (i.e where the 
full byte is used and the string and the integer have the same actual 
content if on appropriate endian machine), which would make the bit 
extraction comparable easy and effective as the i.__hex__() based method.
Using digits() instead of the code you have provided speeds the whole 
thing up two times (see attached code for details), but still is 
i.__hex__() the best way to go and could be improved probably only by 
direct conversion to base 256 or even higher base as e.g. 2**16 or 2**32.

(see Paul Rubin's post first).
BTW, I'd bet that '%x'.__mod__(i) will save you time over i.__hex__() by the 
time
you finish fiddling with 0x and L, even if you don't use the rest.

The question looming is, What are you intending to do with your new 
representations
of twos complement integers?

BTW2, an easy way to play with integers chopped into little-endian bit field 
sequences is
a generator, e.g., (note that last field is sign bits, either all zeroes or all 
ones, so
you can unambiguously reconstruct a signed integer from this variable width 
representation).
(not very tested, and BTW as you see I hacked in using int when possible for 
return values)

  def bitsof(n, width=8):
 ... m = (2**(width-1)-1)*2+1 # allow 2**width-1 == sys.maxint as int
 ... if type(m) is long:
 ... yield nm
 ... while n0 or n-1:
 ... n=width
 ... yield nm
 ... else:
 ... yield int(nm)
 ... while n0 or n-1:
 ... n=width
 ... yield int(nm)
 ...
  list(bitsof(11,1))
 [1, 1, 0, 1, 0]
  list(bitsof(5,1))
 [1, 0, 1, 0]
  list(bitsof(-5,1))
 [1, 1, 0, 1]
  hex(3**100)
 '0x5A4653CA673768565B41F775D6947D55CF3813D1L'
  ''.join(map('%02X'.__mod__, bitsof(3**100, 8))[::-1])
 '005A4653CA673768565B41F775D6947D55CF3813D1'
  ''.join(map('%02X'.__mod__, bitsof(-3**100, 8))[::-1])
 'FFA5B9AC3598C897A9A4BE088A296B82AA30C7EC2F'
  hex(-3**100)
 '-0x5A4653CA673768565B41F775D6947D55CF3813D1L'

gentle rant
(I leave it to your judgement as to how useful our current hex() representation
of negative numebers is ;-)
/gentle rant

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: itertools.izip brokeness

[Bengt Richter]

On 5 Jan 2006 14:34:39 -0800, [EMAIL PROTECTED] wrote:


Bengt Richter wrote:
 On 5 Jan 2006 15:48:26 GMT, Antoon Pardon [EMAIL PROTECTED] wrote:

 On 2006-01-04, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
 [EMAIL PROTECTED] wrote:
  But here is my real question...
  Why isn't something like this in itertools, or why shouldn't
  it go into itertools?
 
 
4) If a need does arise, it can be met by __builtins__.map() or by
   writing:  chain(iterable, repeat(None)).
 
  Yes, if youre a python guru.  I don't even understand the
  code presented in this thread that uses chain/repeat,
 
 And it wouldn't work in this case. chain(iterable, repeat(None))
 changes your iterable into an iterator that first gives you
 all elements in the iterator and when these are exhausted
 will continue giving the repeat parameter. e.g.
 
   chain([3,5,8],repeat(Bye)
 
 Will produce  3, 5 and 8 followed by an endless stream
 of Bye.
 
 But if you do this with all iterables, and you have to
 because you don't know which one is the smaller, all
 iterators will be infinite and izip will never stop.

 But you can fix that (only test is what you see ;-) :

   from itertools import repeat, chain, izip
   it = iter(lambda z=izip(chain([3,5,8],repeat(Bye)), 
 chain([11,22],repeat(Bye))):z.next(), (Bye,Bye))
   for t in it: print t
  ...
  (3, 11)
  (5, 22)
  (8, 'Bye')

 (Feel free to generalize ;-)

Which just reinforces my original point: if leaving
out a feature is justified by the existence of some
alternate method, then that method must be equally
obvious as the missing feature, or must be documented
as an idiom.  Otherwise, the justification fails.

Is the above code as obvious as
  izip([3,5,8],[11,22],sentinal='Bye')?
(where the sentinal keyword causes izip to iterate
to the longest argument.)

You are right. I was just responding with a quick fix to the
problem Antoon noted.
For a more flexible izip including the above capability, but
also abble to do the default izip with a capability of continuing iteration
in the above mode after the normal izip mode stops, see izip2.py in my other
post in this thread.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: inline function call

[Bengt Richter]

On Thu, 05 Jan 2006 08:47:37 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote:

On Wed, 04 Jan 2006 13:18:32 +0100, Riko Wichmann wrote:

 hi everyone,
 
 I'm googeling since some time, but can't find an answer - maybe because 
 the answer is 'No!'.
 
 Can I call a function in python inline, so that the python byte compiler 
 does actually call the function, but sort of inserts it where the inline 
 call is made? Therefore avoiding the function all overhead.

The closest thing to that is the following:


# original version:

for i in xrange(10):
myObject.something.foo()  # three name space lookups every loop


# inline version:

# original version:

foo = myObject.something.foo
for i in xrange(10):
foo()  # one name space lookup every loop

But that isn't in-lining, that's hoisting some expression-evaluation
out of a loop on the assumption that the expression has constant value
and no side effects. That's good optimization, but it isn't in-lining.
Inlining is putting code to accomplish what foo does in place of the foo call.
E.g. if under simple conditions and assumptions you in-lined

def foo(x,y):return 2*x+y+1
in
a = 3*foo(b,c)

the code would be

a = 3*(2*b+c+1)

This could be achieved by a custom import function that would capture the AST
and e.g. recognize a declaration like __inline__ = foo, bar followed by defs
of foo and bar, and extracting that from the AST and modifying the rest of the
AST wherever foo and bar calls occur, and generating suitable substitutions of
suitable AST subtrees generated from the ASTs of the foo and bar ASTs and rules
for renaming and guaranteeing safe temporary names etc. The modified AST would
then pass through the rest of the process to compilation and execution for the
creation of a module. You just wouldn't be able to plain old import to do the
importing (unless you had the custom import hooked in, and assumed that sources
without __inline__ = ... statements would not occur unintended.

Without hooking, usage might look like

import inliner
mymod = inliner.__import__('mymod') # side effect to mymod.pyc and 
sys.modules

and then you could expect calls to designated and defined inline functions in 
mymod.py
to have been inlined in the code of mymod.

I've been meaning to do a proof of concept, but I've hinted at these things 
before,
and no one seems much interested. And besides it's really a distraction from 
more radical
stuff I'd like to try ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is 'everything' a refrence or isn't it?

[Bengt Richter]

On 4 Jan 2006 10:54:17 -0800, KraftDiner [EMAIL PROTECTED] wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
   if i==2:
  i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
   if lst[i] == 2:
  lst[i]=4
print lst

Have I misunderstood something?

Evidently, as others have already elaborated upon ;-)

I am wondering if it might be helpful to focus on expressions
first, and point out that the result is always a reference to
an object representing the value of the expression.

Since you always get a reference to a value-representing object,
assignment can only have meaning in terms of what you can do with that 
reference.
You can never assign a value per se. Therefore an assignment is
placing the reference in a place determined by the assignment target syntax.

The simplest is a bare name, and then we speak of binding a name to the
value-representing object. The name belongs to some context, and name space,
and an implementation will determine how to find the named binding site that
will hold the reference, depending on what name space is involved. E.g. the
binding site might be an optimized slot in a function's local name space.

Other assignment syntaxes also create bindings, and also identify what I am
calling binding sites. These sites may be within the representation of another 
object,
and located by some computation implementing the semantics, such as finding
an appropriate offset into the current representation of a list, or a 
hash-accessed
bucket/slot of a dict etc.

BTW, the implementation of finding a binding site for a non-bare-name binding
is typically a method call with the method and arguments determined by the 
trailing
end of the assignment target expression and the object whose method is to be 
found
determined by the expression up to the trailing end.

Binding site implementation is not part of the language, but the semantics of
binding objects 1:1 or 1:many is.

Mutable objects have assignable binding sites. Immutable objects do not (except
for the assignment of initial binding references during their creation). 
Containing objects
contain binding sites, not values. Binding sites must have references to some
value-representing object or other if they semantically exist, though of course 
an implementation
may manage space any way convenient if semantics are preserved.

Going back to an expression again, when e.g., a name is evaluated, the name 
identifies
a binding site somwhere, and the value of any expression (including bare name) 
is, as we said at the start,
a reference to an object representing the value of the expression.

In the case of a bare name, we have that reference as soon as we have 
identified the binding site
associated with the name, since that contains a reference to a 
value-representing object. If this reference
is now assigned to a binding site identified by another bare name, we have two 
bare names with respective
associated binding sites having references referring to the same 
value-representing object.

IOW, python shares value representations by default. Including mutable value 
representations. Which
means if you want a distinct mutable value equal to a given one, you need an 
expression that does
more than evaluate to find the reference to the given one -- i.e., creates a 
new value-representing object
using the given for information, and returns a reference the new.

To a C programmer, it will be tempting to say that binding sites are just 
pointer-data locations,
and references are pointers, and value-representing-object is just malloc'ed 
space for some custom
struct or array or both. This could be true for a particular implementation, 
but I think binding site
and reference as I have just used them, are also abstract concepts, capable 
of alternative, non-C
implementation of the python semantics. So I thought I'd try this as a possible 
terminology, to see if
it makes it any easier for anyone ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Hypergeometric distribution

[Bengt Richter]

On Thu, 05 Jan 2006 09:47:02 -0600, Robert Kern [EMAIL PROTECTED] wrote:

Bengt Richter wrote:
 On 4 Jan 2006 12:46:47 -0800, Raven [EMAIL PROTECTED] wrote:

The problem with Stirling's approximation is that I need to calculate
the hypergeometric hence the factorial for numbers within a large range
e.g. choose(14000,170) or choose(5,2)
 
 It seems you are hinting at some accuracy requirements that you haven't
 yet explained. I'm curious how you use the values, and how that affects your
 judgement of Stirling's approximation. In fact, perhaps the semantics of your
 value usage could even suggest an alternate algorithmic approach to your 
 actual end result.

Does it matter? Implementing Stirling's approximation is pointless when
scipy.special.gammaln() or scipy.special.gamma() does it for him.

Who's talking about implementing Stirling's approximation? ;-) I'm trying to 
determine first
why the OP is thinking there's a problem with using it at all. With alternate 
algorithmic
approach I didn't mean an alternate way of calculating Stirling's 
approximation. I meant
to allude to the possibility that pulling a little further on the requirements 
thread might
even unravel some of the rationale for calculating the hypergeometric per se, 
depending on
how he's actually using it and why. Same old, same old: requirements, 
requirements ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: itertools.izip brokeness

[Bengt Richter]

On 5 Jan 2006 15:48:26 GMT, Antoon Pardon [EMAIL PROTECTED] wrote:

On 2006-01-04, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
[EMAIL PROTECTED] wrote:
 But here is my real question...
 Why isn't something like this in itertools, or why shouldn't
 it go into itertools?


   4) If a need does arise, it can be met by __builtins__.map() or by
  writing:  chain(iterable, repeat(None)).

 Yes, if youre a python guru.  I don't even understand the
 code presented in this thread that uses chain/repeat,

And it wouldn't work in this case. chain(iterable, repeat(None))
changes your iterable into an iterator that first gives you
all elements in the iterator and when these are exhausted
will continue giving the repeat parameter. e.g.

  chain([3,5,8],repeat(Bye)

Will produce  3, 5 and 8 followed by an endless stream
of Bye.

But if you do this with all iterables, and you have to
because you don't know which one is the smaller, all
iterators will be infinite and izip will never stop.

But you can fix that (only test is what you see ;-) :

  from itertools import repeat, chain, izip
  it = iter(lambda z=izip(chain([3,5,8],repeat(Bye)), 
  chain([11,22],repeat(Bye))):z.next(), (Bye,Bye))
  for t in it: print t
 ...
 (3, 11)
 (5, 22)
 (8, 'Bye')

(Feel free to generalize ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: itertools.izip brokeness

[Bengt Richter]

On Thu, 05 Jan 2006 07:42:25 GMT, [EMAIL PROTECTED] (Bengt Richter) wrote:

On 4 Jan 2006 15:20:43 -0800, Raymond Hettinger [EMAIL PROTECTED] wrote:

[ ... 5 options enumerated ... ]


6. Could modify izip so that one could write

from itertools import izip
zipit = izip(*seqs)# bind iterator object to preserve access to 
 its state later
for tup in zipit:
# do something with tup as now produced
for tup in zipit.rest(sentinel):
# tup starts with the tuple that would have been returned if all 
 sequences
# had been sampled and sentinel substituted where StopIteration 
 happened.
# continuing until but not including (sentinel,)*len(seqs)

This would seem backwards compatible, and also potentially allow you to use 
the rest mode
from the start, as in

for tup in izip(*seqs).rest(sentinel):
# process tup and notice sentinel for yourself

Demo-of-concept hack: only tested as you see below

 izip2.py -
class izip2(object):

works like itertools.izip except that
if a reference (e.g. it) to the stopped iterator is preserved,
it.rest(sentinel) returns an iterator that will continue
to return tuples with sentinel substituted for items from
exhausted sequences, until all sequences are exhausted.

FIRST, FIRST_STOP, FIRST_REST, REST, REST_STOP = xrange(5)
def __init__(self, *seqs):
self.iters = map(iter, seqs)
self.restmode = self.FIRST
def __iter__(self): return self
def next(self):
if not self.iters: raise StopIteration
if self.restmode == self.FIRST:
tup=[]
try:
for i, it in enumerate(self.iters):
tup.append(it.next())
return tuple(tup)
except StopIteration:
self.restmode = self.FIRST_STOP # stopped, not rest-restarted
self.tup=tup;self.i=i
raise
elif self.restmode==self.FIRST_STOP:  # normal part exhausted
raise StopIteration
elif self.restmode in (self.FIRST_REST, self.REST):
if self.restmode == self.FIRST_REST:
tup = self.tup # saved
self.restmode = self.REST
else:
tup=[]
for it in self.iters:
try: tup.append(it.next())
except StopIteration: tup.append(self.sentinel)
tup = tuple(tup)
if tup==(self.sentinel,)*len(self.iters):
self.restmode = self.REST_STOP
raise StopIteration
return tuple(tup)
elif self.restmode==self.REST_STOP:  # rest part exhausted
raise StopIteration
else:
raise RuntimeError('Bad restmode: %r'%self.restmode)
def rest(self, sentinel=''):
self.sentinel = sentinel
if self.restmode==self.FIRST: # prior to any sequence end
self.restmode = self.REST
return self
self.tup.append(sentinel)
for it in self.iters[self.i+1:]:
try: self.tup.append(it.next())
except StopIteration: self.tup.append(sentinel)
self.restmode = self.FIRST_REST
return self

def test():
assert list(izip2())==[]
assert list(izip2().rest(''))==[]
it = izip2('ab', '1')
assert list(it)==[('a', '1')]
assert list(it.rest())==[('b', '')]
it = izip2('a', '12')
assert list(it)==[('a', '1')]
assert list(it.rest())==[('', '2')]
it = izip2('ab', '12')
assert list(it)==[('a', '1'), ('b', '2')]
assert list(it.rest())==[]
it = izip2(xrange(3), (11,22), 'abcd')
assert list(it) == [(0, 11, 'a'), (1, 22, 'b')]
assert list(it.rest(None)) == [(2, None, 'c'), (None, None, 'd')]
print 'test passed'

if __name__ == '__main__': test()
-

Using this, Antoon's example becomes:

  from izip2 import izip2
  it = izip2([3,5,8], [11,22])
  for t in it: print t
 ...
 (3, 11)
 (5, 22)
  for t in it.rest('Bye'): print t
 ...
 (8, 'Bye')

Want to make an efficient C version, Raymond? ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Translate this to python?

[Bengt Richter]

On 3 Jan 2006 17:42:44 -0800, KraftDiner [EMAIL PROTECTED] wrote:

I'm porting a routing from C++ to python.
There is a complex for loop that I don't know how to code in python

for (i = nPoints-1, j = 0; j  nPoints; i = j, j++)

Perhaps most directly comparable semantics (untested):

i = nPoints-1
j = 0
while jnPoints:
# whatever body code
i = j
j += 1

Not sure what i is really for, but j seems to be independent,
so perhaps (also untested, and caveat: it's past bedtime)

i = nPoints - 1
for j in xrange(nPoints):
# whatever
i = j

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Hypergeometric distribution

[Bengt Richter]

On 4 Jan 2006 12:46:47 -0800, Raven [EMAIL PROTECTED] wrote:

Cameron Laird wrote:

 This thread confuses me.

 I've lost track of the real goal.  If it's an exact calculation of
 binomial coefficients--or even one of several other potential
 targets mentioned--I echo Steven D'Aprano, and ask, are you *sure*
 the suggestions already offered aren't adequate?

Hi Cameron, my real goal was to calculate the hypergeometric
distribution. The problem was that the  function for hypergeometric
ISTM that can't have been your real goal -- unless you are e.g. preparing 
numeric
tables for publication. IOW, IWT you probably intend to USE the hypergeometric
distribution values in some useful way to go towards your real goal. ;-)

The requirements of this USE are still not apparent to me in your posts, though
that may be because I've missed something.

calculation from scipy uses the scipy.comb function which by default
uses floats so for large numbers comb(n,r) returns inf. and hence the
hypergeometric returns nan.
The first suggestion, the one by Robert Kern,  resolved my problem:

Raven wrote:
Thanks to all of you guys, I could resolve my problem using the
logarithms as proposed by Robert.

Then the other guys gave alternative solutions so I tried them out. So
form me the suggestions offered are more than adequate :-)

Cameron Laird wrote:
Also, I think you
 might not realize how accurate Stirling's approximation (perhaps to
 second order) is in the range of interest.

The problem with Stirling's approximation is that I need to calculate
the hypergeometric hence the factorial for numbers within a large range
e.g. choose(14000,170) or choose(5,2)

It seems you are hinting at some accuracy requirements that you haven't
yet explained. I'm curious how you use the values, and how that affects your
judgement of Stirling's approximation. In fact, perhaps the semantics of your
value usage could even suggest an alternate algorithmic approach to your actual 
end result.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: itertools.izip brokeness

[Bengt Richter]

On 4 Jan 2006 15:20:43 -0800, Raymond Hettinger [EMAIL PROTECTED] wrote:

Paul Rubin wrote:
 What do you think of my suggestion of passing an optional arg to the
 StopIteration constructor, that the caller can use to resume the
 iterator or take other suitable recovery steps?  Maybe this could
 interact with PEP 343 in some useful way.

Probably unworkable.  Complex to explain and use.   Makes the API
heavy.  Hard to assure retro-fitting for every possible kind of
iterator or iterable.

Am not sure of the best solution:

1. Could add an optional arg to zip()/izip() with a mutable container
to hold a final, incomplete tuple:  final=[]; zip(a,b,leftover=final).
This approach is kludgy and unlikely to lead to beautiful code, but it
does at least make accessible data that would otherwise be tossed.

2. Could add a new function with None fill-in -- essentially an
iterator version of map(None, a,b).  Instead of None, a user specified
default value would be helpful in cases where the input data stream
could potentially have None as a valid data element.  The function
would also need periodic signal checks to make it possible to break-out
if one the inputs is infinite. How or whether such a function would be
used can likely be answered by mining real-world code for cases where
map's None fill-in feature was used.

3. Could point people to the roundrobin() recipe in the
collections.deque docs -- it solves a closely related problem but is
not exactly what the OP needed (his use case required knowing which
iterator gave birth to each datum).

4. Could punt and leave this for straight-forward while-loop coding.
Though the use case seems like it would be common, there may be a
reason this hasn't come up since zip() was introduced way back in
Py2.0.

5. Could create an iterator wrapper that remembers its last accessed
item and whether StopIteration has been raised.  While less direct than
a customized zip method, the wrapper may be useful in contexts other
than zipping -- essentially, anywhere it is inconvenient to have just
consumed an iterator element.  Testing the wrapper object for
StopIteration would be akin to else-clauses in a for-loop. OTOH, this
approach is at odds with the notion of side-effect free functional
programming and the purported benefits of that programming style.


6. Could modify izip so that one could write

from itertools import izip
zipit = izip(*seqs) # bind iterator object to preserve access to its state 
later
for tup in zipit:
# do something with tup as now produced
for tup in zipit.rest(sentinel):
# tup starts with the tuple that would have been returned if all 
sequences
# had been sampled and sentinel substituted where StopIteration 
happened.
# continuing until but not including (sentinel,)*len(seqs)

This would seem backwards compatible, and also potentially allow you to use the 
rest mode
from the start, as in

for tup in izip(*seqs).rest(sentinel):
# process tup and notice sentinel for yourself


Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Hypergeometric distribution

[Bengt Richter]

On 2 Jan 2006 03:35:33 -0800, Raven [EMAIL PROTECTED] wrote:
[...]

The problem with long integers is that to calculate the hypergeometric
I need to do float division and multiplication because integer division
returns 0. A solution could be to calculate log(Long_Factorial_Integer)

ISTM you wouldn't get zero if you scaled by 10**significant_digits (however many
you require) before dividing. E.g., expected hits per trillion (or septillion 
or whatever)
expresses probability too. Perhaps that could work in your calculation?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python coding contest

[Bengt Richter]

On 27 Dec 2005 09:24:44 GMT, Duncan Booth [EMAIL PROTECTED] wrote:

Scott David Daniels wrote:

 I definitively need a new algorythm. g
 
 And I am sadly stuck at 169.  Not even spitting distance from 149 (which
 sounds like a non-cheat version).

Throw it away and start again with a fresh (clean) solution. That's what I 
did when I'd reached the limit of nested maps and lambdas at 150 
characters. I'm now on 134 characters and the solution is very nearly 
legible. (Frustratingly, I'm away for the next few days, so I may not get a 
chance to submit my solution).

It would be a nice idea to come up with a scoring system which better 
reflects Python's ideals. For example, use the parser in Python to count up 
various syntactic elements, score 0 for comments, indents, dedents, 
newlines, docstrings, 1 for each name or operation used and higher scores 
for things like lambda or overly complex expressions.

[23:28] C:\pywk\clp\seven\pycontest_01py24 test.py
.
--
Ran 1 test in 0.391s

OK

[23:28] C:\pywk\clp\seven\pycontest_01wc -lc  seven_seg.py
  2136  seven_seg.py

2 lines, 136 chars including unix-style lineseps (is that cheating on windows?)
No imports.

Guess I'll have to try another tack ;-/

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python coding contest

[Bengt Richter]

On Sun, 25 Dec 2005 16:39:47 +0100, Simon Hengel [EMAIL PROTECTED] wrote:

Hello,
we are hosting a python coding contest an we even managed to provide a
price for the winner...
 ^

How much are you going to sell him or her for? ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Windows and python execution

[Bengt Richter]

On Mon, 26 Dec 2005 14:53:28 +, Mark Carter [EMAIL PROTECTED] wrote:

rzed wrote:
 Mark Carter [EMAIL PROTECTED] wrote in
 news:[EMAIL PROTECTED]: 
 
 
What I would like to do it type something like

myscript.py

instead of

python myscript.py

 As another poster points out, be sure that your Python is on your 
 path. 
 
 And there is a PATHEXT environment variable, 

Aha. You'bve provided a significant clue.

What you need to do is include the following line in autoexec.bat:
set .py=c:\python24\python.exe

This will achieve the desired result. I'm suprised more people don't use it.
I wasn't aware of that syntax for set. What OS/platform/shell is that from?
How did you go from the PATHEXT clue to the set command you specify and decide
not to set pathext, e.g., by something like
set PATHEXT=%PATHEXT%;.py
Does your set do the pathext and assoc and ftype all in one swell foop?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido at Google

[Bengt Richter]

On 22 Dec 2005 23:06:43 -0800, Anand [EMAIL PROTECTED] wrote:

My newsreader automatically (and configurably) generates the above line.
Has a new reader come into frequent use that by default does not?
ISTM that I've seen a lot of unattributed quotes posted recently.

 It's like having James Bond as your very own personal body guard ;)

That is such a nice quote that I am going to put it in my email
signature ! :)

-Anand

Maybe look into fixing the above problem while you're at it?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging: Formatter: name of the function

[Bengt Richter]

On Fri, 23 Dec 2005 16:23:57 +0100, Gregor Horvath [EMAIL PROTECTED] wrote:

Hi,

Is there a possibility to format a log message to give the function name 
where the log appears?

Example

import logging

def aTestFunction():
   logger.debug(This is a message)

The log should read:

aTestFunction  This is a message.

There is a possibilty to format the module namewith %(module)s, but I 
did not find one for the function name.

Is there a possibilty or can I create it myself?
How can I determine the function name within a function?
Introspection?

There's not a nice way that I know of, but you can do something like

  import sys
  def foo():
 ... print 'logger string containing function name 
%s'%sys._getframe().f_code.co_name
 ...
  foo()
 logger string containing function name foo

However, you might want to consider using a decorator that could
wrap a function if debugging and leave it alone otherwise. Also you
can log before and after calling the function. E.g.,

  def debugdeco(f):
 ... if not __debug__: return f
 ... def wrap(*args, **kw):
 ... print 'before entering function %s ...'%f.func_name
 ... result = f(*args, **kw)
 ... print 'after returning from function %s ...'%f.func_name
 ... return result
 ... wrap.func_name = f.func_name # make it look the same if desired
 ... return wrap
 ...
  @debugdeco
 ... def foo(something): print something; return 'whatever'
 ...
  foo('hello')
 before entering function foo ...
 hello
 after returning from function foo ...
 'whatever'
  __debug__ = False
   File stdin, line 1
 SyntaxError: can not assign to __debug__

Oops, can't experiment that way ;-)
  ^Z

We'll just start it with -O to set __debug__ False:
 
 [ 8:45] E:\Info\Politicspy24 -O
 Python 2.4b1 (#56, Nov  3 2004, 01:47:27)
 [GCC 3.2.3 (mingw special 20030504-1)] on win32
 Type help, copyright, credits or license for more information.
  def debugdeco(f):
 ... if not __debug__: return f
 ... def wrap(*args, **kw):
 ... print 'before entering function %s ...'%f.func_name
 ... result = f(*args, **kw)
 ... print 'after returning from function %s ...'%f.func_name
 ... return result
 ... wrap.func_name = f.func_name # make it look the same if desired
 ... return wrap
 ...
  @debugdeco
 ... def foo(something): print something; return 'whatever'
 ...
  foo('hello')
 hello
 'whatever'

You could still do stuff unconditionally of course, and also inside foo.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido at Google

[Bengt Richter]

On Wed, 21 Dec 2005 21:47:29 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

Bengt Richter wrote:
[roughly an inch is not exactly 25.4mm]
 At least according to my dusty 37th Edition Handbook of Chemistry and 
 Physics (c) 1955.
 Maybe things have changed since then ;-)

Wikipedia concurs with Jim, though it says the official change dates 
from 1958.

Better throw that old book out, as it's also likely to be missing any 
reference to useful elements such as Lawrencium (1961), and Hassium 
(1984), not to mention Ununnilium, Ununumium and Ununbium (94, 94, 96 
respectively) or the most recently discovered element, which the PSU 
tried to supp

I had been using 25.4mm/inch myself, but decided to look it up, and
found that I had been using the wrong value -- now actually proving
to be right after all, after the definition change of 1958(1959?).

Google found an NIST page:

http://physics.nist.gov/Pubs/SP811/appenB.html

Where it says:

B.6 U.S. survey foot and mile

The U. S. Metric Law of 1866 gave the relationship 1 m = 39.37 in (in is
the unit symbol for the inch). From 1893 until 1959, the yard was defined
as being exactly equal to (3600/3937) m, and thus the foot was defined as
being exactly equal to (1200/3937) m.

In 1959 the definition of the yard was changed to bring the U.S. yard and
the yard used in other countries into agreement. Since then the yard has
been defined as exactly equal to 0.9144 m, and thus the foot has been
defined as exactly equal to 0.3048 m. At the same time it was decided that
any data expressed in feet derived from geodetic surveys within the United
States would continue to bear the relationship as defined in 1893, namely,
1 ft = (1200/ 3937) m (ft is the unit symbol for the foot). The name of
this foot is U.S. survey foot, while the name of the new foot defined in
1959 is international foot. The two are related to each other through
the expression 1 international foot = 0.999 998 U.S. survey foot exactly.

In Sec. B.8 and Sec. B.9, the factors given are based on the international
foot unless otherwise indicated. Users of this /Guide/ may also find
the following summary of exact relationships helpful, where for
convenience the symbols /ft/ and /mi,/ that is, ft and mi in
italic type, indicate that it is the /U.S. survey foot/ or /U.S.
survey mile/ that is meant rather than the international foot (ft) or
international mile (mi), and where rd is the unit symbol for the rod and
fur is the unit symbol for the furlong.

1 /ft/ = (1200/3937) m 
1 ft = 0.3048 m 
1 ft = 0.999 998 /ft/ 
1 rd, pole, or perch = 16 1/2 /ft/ 

40 rd = 1 fur = 660 /ft/ 
8 fur = 1 U.S. survey mile (also called statute mile) = 1 /mi/ = 5280 /ft/
1 fathom = 6 /ft/ 
1 international mile = 1 mi = 5280 ft 
272 1/4 /ft/**2 = 1 rd**2  

160 rd**2 = 1 acre = 43 560ft**2 
640 acre = 1 /mi/**2 

(I changed italics to be indicated by /italic/ slashes, and superscript by **,
as well as changing special characters for a quarter and half to 1/4 and 1/2.
Hope I didn't typo ;-)

Anyway, 25.4 mm/inch it is. Nice to revert to that, after an unsettling 
diversion ;-)
NIST ought to have it right, right? Or is there an intelligent design version 
now? ;-/

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido at Google

[Bengt Richter]

On Wed, 21 Dec 2005 18:30:03 -0700 (MST), Jim Benson [EMAIL PROTECTED] wrote:

On Thu, 22 Dec 2005, Bengt Richter wrote:

  
  For Americans: 15 meters is roughly 50 feet.
 
 Google can do that too, of course. wink
 
 http://www.google.com/search?q=convert+15+meters+to+feet
 
 (49.2125984 feet to be more precise)
 
 Actually that looks like it's based on the approximation
 of 25.4 mm/inch, whereas I believe the legally defined US conversion
 is 39.3700 inches/meter. They're close. British is 39.3701 for some reason.
 At least according to my dusty 37th Edition Handbook of Chemistry and 
 Physics (c) 1955.
 Maybe things have changed since then ;-)
 

Actually they did change...My 54th edition lists the change that
as of July 1 1959, by definition, 1 inch is exactly 25.4 mm.
 
So I found, which makes me happy, because I had been assuming 25.4
for a long time. (I'm not happy about saying I believe the legally
defined conversion... since I had only just tried to verify 25.4
and found that I was wrong. I guess something told me to hedge with that 
maybe ;-)

FWIW, my first reference was my trusty old Random House American College 
Dictionary
(that I used in high school) dictionary, which also says 39.37 in/meter.
But it's copyrighted 1949. They used to make real reference books with good 
paper ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: deal or no deal

[Bengt Richter]

On Thu, 22 Dec 2005 09:29:49 -0500, rbt [EMAIL PROTECTED] wrote:

The house almost always wins or are my assumptions wrong...

import random

amounts = [.01, 1, 5, 10, 25, 50, 75, 100, 200, 300, 400, 500, 750,
1000, 5000, 1, 25000, 5, 75000, 10, 20,
30, 40, 50, 75, 100]

results = []

count = 0
while count  10:
 count = count + 1
 pick = random.choice(amounts)
 if pick  10:
 results.append(NBC won... Your briefcase contains $%s %pick)
 else:
 results.append(You won... Your briefcase contains $%s %pick)

results.sort()
print Here are 10 random picks: 
for result in results:
 print result

I don't know what you are doing, but 10 is a small sample to draw confident
conclusions from. E.g., try counting how many times pick10 out of
a larger number, e.g.,

(BTW, the expression xy has the value True or False, and True and False are
int subclass instances with int values 1 and 0 respectively, so the sum below
is adding 1 whenever random.choice(amounts)10 and 0 otherwise)

  import random
  amounts = [.01, 1, 5, 10, 25, 50, 75, 100, 200, 300, 400, 500, 750,
 ... 1000, 5000, 1, 25000, 5, 75000, 10, 20,
 ... 30, 40, 50, 75, 100]
  sum(random.choice(amounts)10 for _ in xrange(1000))
 736
  sum(random.choice(amounts)10 for _ in xrange(1000))
 734
  sum(random.choice(amounts)10 for _ in xrange(1000))
 730
  sum(random.choice(amounts)10 for _ in xrange(1000))
 721
  sum(random.choice(amounts)10 for _ in xrange(1000))
 753

What should the sum be?

  amounts.index(10)
 19
is the number of numbers  10 in amounts

  len(amounts)
 26
is the total number of numbers

So if you choose with uniform probability, you'd expect
  19./26
 0.73076923076923073

to be the fraction satisfying pick10

With a larger sample, the fraction should show better, e.g.,

  sum(random.choice(amounts)10 for _ in xrange(100))
 730983

Does that help?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido at Google

[Bengt Richter]

On Thu, 22 Dec 2005 09:07:26 -0800, [EMAIL PROTECTED] (Alex Martelli) wrote:

Renato [EMAIL PROTECTED] wrote:

 all of the native administration tools of RedHat (all versions) and
 Fedora Core are written in python (system-config-* and/or
 redhat-config-* ). And even more importantly, yum (the official
 software package manager for Fedora and RHEL) and Anaconda (OS
 installer) are written in Python, too.

BTW, Chip Turner (from RedHat, and deeply involved in those
developments) happened to start at Google the same day I did;-).

So is google about to determine how many luminaries can fit on the head of a 
project? ;-)
Seriously, if you heavies do sometimes work on the same project, it would be
interesting to know what modes of co-operation you tend to adopt.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Providing 'default' value with raw_input()?

[Bengt Richter]

On 22 Dec 2005 08:55:17 -0800, planetthoughtful [EMAIL PROTECTED] wrote:

Hi All,

As always, my posts come with a 'Warning: Newbie lies ahead!'
disclaimer...

I'm wondering if it's possible, using raw_input(), to provide a
'default' value with the prompt?
Sure, depending on what you mean ;-)
You can only get a prompt string printed to the screen with raw_input,
but you can build that string any way you want, so it can show e.g.,
User-friendly prose and the current value and a default value and anything else.

Then the user has to enter zero or more characters and press Enter.
Whether something changes value or not, and what value is used, depends on
what the program does with the string (possibly empty) that the user typed.

So you have to decide what user response should mean change old value to 
default as shown
and what should mean make no change and what should mean convert the typed 
string to a value
and change the old value to that.

To have a user-friendly program, you won't want to make user errors too easy, 
and you
will probably want to have the program re-prompt if there is an input typo (or 
maybe
an input that converts ok but has bad business consequences (e.g., like that 
recent
Japanese stock sale switcheroo of price and number of shares that cost many 
megabucks).


I would like to include the ability to edit an existing value (drawn
from an SQLite table) using a DOS console Python app, but my gut
feeling from reading what I can find about raw_input() is that it only
allows you to provide a prompt, not a default value as well.

What's wrong with showing the default in the prompt itself? The thing is
what user input in response should signify the default? E.g., maybe a single
period by itself could mean use the default and nothing could mean no change
and other input would be a new value.

If anyone can give me any advice on how I might achieve this, I would
be immensely appreciative!

suppose you had some named values in one dict, e.g.,

  for name, price in values.items():
 ... while True: # retry until satisfied
 ... uresp = raw_input(
 ... '\n'
 ... 'Current price of %s is $%.2f %s\n'
 ... 'Press Enter to leave as is, or\n'
 ... 'type D (without quotes) followed by Enter to change price 
to %.2f,\n'
 ... 'or enter a new value: ' % (name, price, info[name][0], 
info[name][1]))
 ... if not uresp: break
 ... if uresp=='D': values[name] = info[name][1]; break
 ... else:
 ... try: values[name] = type(values[name])(uresp); break
 ... except Exception, e:
 ... print '%s: %s'%(e.__class__.__name__, e)
 ... print 'Try again'
 ...
 
 Current price of yoghurt is $0.90 each
 Press Enter to leave as is, or
 type D (without quotes) followed by Enter to change price to 1.19,
 or enter a new value: D
 
 Current price of banana is $0.79 per pound
 Press Enter to leave as is, or
 type D (without quotes) followed by Enter to change price to 0.99,
 or enter a new value:
 
 Current price of apple is $1.29 per pound
 Press Enter to leave as is, or
 type D (without quotes) followed by Enter to change price to 1.49,
 or enter a new value: 1..23
 ValueError: invalid literal for float(): 1..23
 Try again
 
 Current price of apple is $1.29 per pound
 Press Enter to leave as is, or
 type D (without quotes) followed by Enter to change price to 1.49,
 or enter a new value: 1.23
 

Ok, now we can see what happened:

  for name, price in values.items(): print '%10s: %.2f'%(name,price)
 ...
yoghurt: 1.19
 banana: 0.79
  apple: 1.23
 
I'm not suggesting this as a pattern to follow, just to illustrate what you can 
do
with raw_input. IOW, it's about how you build the prompt string, and what you do
with the user response input string that solves your problem.

Voluminous prompts get old fast, so you might e.g. want to accept a '?' to get 
extra
context-sensitive info instead, followed by retry.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: jython: True and False boolean literals?

[Bengt Richter]

On 22 Dec 2005 10:51:22 -0800, [EMAIL PROTECTED] wrote:

Aren't there boolean literals for True and False in Python
(jython)?  I can't get true, True, false, or False to work.  I
ended up having to use (1==1) and (1==0).

You may want to upgrade to a newer version.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Providing 'default' value with raw_input()?

[Bengt Richter]

On 22 Dec 2005 12:42:36 -0800, planetthoughtful [EMAIL PROTECTED] wrote:


Bengt Richter wrote:
 On 22 Dec 2005 08:55:17 -0800, planetthoughtful [EMAIL PROTECTED] wrote:


 I would like to include the ability to edit an existing value (drawn
 from an SQLite table) using a DOS console Python app, but my gut
 feeling from reading what I can find about raw_input() is that it only
 allows you to provide a prompt, not a default value as well.
 
 What's wrong with showing the default in the prompt itself? The thing is
 what user input in response should signify the default? E.g., maybe a single
 period by itself could mean use the default and nothing could mean no 
 change
 and other input would be a new value.

Hi Bengt,

I think that would be a great solution if the value being 'edited' was
relatively short, or numeric, etc. But the values I want to 'edit' can
be several hundred characters long (ie they're text fields containing
todo notes), and using your method, I'd either accept the existing
value, or have to retype the text for that record. I think I mislead
people when I used the word 'default' -- I meant in the sense of being
able to edit the existing value, so that value, out of each record,
would be presented as the 'default', available for editing.

Sorry for the confusion.

If you want to edit text, maybe the best thing is a text editor?
Why not write it into a temp file and start the user's favorite editor
on it, and when done, read the file back into python and delete the temp.
Maybe check whether any changes actually happened before updating the DB.

os.popen{2,3,4} and tempfile module might help.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: parsing engineering symbols

[Bengt Richter]

On Tue, 20 Dec 2005 23:28:12 +0100, Fredrik Lundh [EMAIL PROTECTED] wrote:

Suresh Jeevanandam wrote:

 I want to convert a string to float value. The string contains
 engineering symbols.

 For example,

 s = '12k'

 I want some function which would return 12000
 function(s)
 = 12000.0
 I searched the web, but could not find any function.

how about:

SI_prefixes = {
'Y':24, 'Z':21, 'E':18, 'P':15, 'T':12, 'G':9, 'M':6, 'k':3,
'h':2, 'd':-1, 'c':-2, 'm':-3, u'\xb5':-6, 'u':-6, 'n':-9, 'p':-12,
'f':-15, 'a':-18, 'z':-21, 'y':-24
}

def myfloat(str):
try:
exp = SI_prefixes[str[-1]]
return float(str[:-1]) * 10**exp
except KeyError:
return float(str)

?

Nit: why not move 10** out of myfloat into SI_prefixes
 (i.e., just retrieve precalculated factors)?

Nit_2: No twinge of conscience shadowing str? ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to calculate number of character in string

[Bengt Richter]

On 21 Dec 2005 15:57:35 +0100, Ove Svensson [EMAIL PROTECTED] wrote:

Ove Svensson [EMAIL PROTECTED] writes:

 P. Schmidt-Volkmar [EMAIL PROTECTED] writes:
 
  Hi there,
  
  I have a string in which I want to calculate how often the character  ';' 
  occurs. If the character does not occur 42 times, the ; should be added 
  so 
  the 42 are reached.
  
  My solution is slow and wrong:
  for Position in range (0, len(Zeile)):
  if Zeile[Position]==';': AnzahlSemikolon = AnzahlSemikolon +1
  if AnzahlSemikolon  42:
  for Zaehler in range(AnzahlSemikolon, 42):
  Zeile = Zeile + ';'
  Dreckskram = Dreckskram +1
  
  How can this be achieved easily?
  
  Thanks,
  
  Pascal 
  
  
 
 What about this:
 
 Zaehler += ';'*max(0,42-Zaehler.count(';'))
 

Sorry, should have been

Zeile += ';'*max(0,42-Zeile.count(';'))

I think You don't need the max

  for n in xrange(-3,4): print '%3s: %r'%(n, n*';')
 ...
  -3: ''
  -2: ''
  -1: ''
   0: ''
   1: ';'
   2: ';;'
   3: ';;;'

I.e.,

Zeile += ';'*(42-Zeile.count(';'))

should work, since a string is a sequence type and

   http://docs.python.org/lib/typesseq.html

Says

Operation   ResultNotes
...
s * n , n * s   n shallow copies of s concatenated(2)
...
(2)
Values of n less than 0 are treated as 0 (which yields an empty sequence of 
the same type as s). ...


I guess it might be nice to mention this in help(str) also, to publish a useful 
fact better.
Maybe under str.__mul__ ?

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reducing import try/except boilerplate

[Bengt Richter]

On Thu, 22 Dec 2005 00:44:52 +1000, Nick Coghlan [EMAIL PROTECTED] wrote:

Several standard library modules (e.g., cPickle/pickle, cStringIO/StringIO, 
threading/dummy_threading) have versions which may not be available on all 
platforms, and pure Python fallbacks that work on any platform Python 
supports. Flicking through the latest version of the Python Cookbook, I 
noticed many recipes that included module fallback suggestions along the lines 
of:

try:
   import cPickle as pickle
except ImportError:
   import pickle

try:
   import threading
except ImportError
   import dummy_threading as threading

That seems rather verbose for something that isn't that uncommon (these 
module all expose the same API, so just give me one of them in this order of 
preference). So what about:

import cPickle or pickle as pickle
import threading or dummy_threading as threading
# 'as' clause required since Python can't guess the name the programmer wants

Also:

   from threading or dummy_threading import Thread
   # No 'as' clause needed since the module name isn't bound


Insomnia-induced-random-ideas-are-fun-'ly yours,
Nick.
Yup. Maybe we should have BIOTW (Best Idea Of The Week) nominations.
I'd say this is a candidate.

The grammar for import stuff is messy though. Have you looked to see
how this would work in? And BTW, I suppose 'or' could be an arbitrarily
long shortcutting 'or' expression? I.e.,

import x or y or z or etc as std_module_name
from x or y or x or etc import std_module_name

Anyway, +1

Hm, not to muddle your idea with featuritis, but how about allowing
string expressions in the place of names, differentiating between
bare names and bare-name-expressions at the same level as import thus:

g = 'gname'
import g as mod   # = mod = __import__('g')
import (g) as mod # = mod = __import__('gname')

h = ''
import g or (h or g) or h as mod # (h or g) is a string expression = 
'gname' here

Also, a bare 'and' could make its predecessor term be treated like an ordinary 
expression (even if
a bare name), allowing bare guard condition expressions, e.g.,

import cond and name or alternate as mod # == import (cond and 'name') or 
alternate as mod

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido at Google

[Bengt Richter]

On Wed, 21 Dec 2005 16:40:15 -0500, Peter Hansen [EMAIL PROTECTED] wrote:

rbt wrote:
 Alex Martelli wrote:
 
I don't think there was any official announcement, but it's true -- he
sits about 15 meters away from me;-).
 
 For Americans: 15 meters is roughly 50 feet.

Google can do that too, of course. wink

http://www.google.com/search?q=convert+15+meters+to+feet

(49.2125984 feet to be more precise)

Actually that looks like it's based on the approximation
of 25.4 mm/inch, whereas I believe the legally defined US conversion
is 39.3700 inches/meter. They're close. British is 39.3701 for some reason.
At least according to my dusty 37th Edition Handbook of Chemistry and Physics 
(c) 1955.
Maybe things have changed since then ;-)

  15e3/25.4/12
 49.212598425196852

Appears to be the google number

But the official conversion

  1000/39.37
 25.400050800101603

is not _exactly_ 25.4 mm/inch
so the distance from Martellibot to BDFL should
more exactly be

  15*39.37/12
 49.2124999

Send bug report to google ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enumeration idioms: Values from different enumerations

[Bengt Richter]

On Tue, 20 Dec 2005 09:16:03 +1100, Ben Finney [EMAIL PROTECTED] wrote:

[EMAIL PROTECTED] writes:
 Ben Finney wrote:
 Is there some behaviour other than evaluate to False or raise an
 exception, that could indicate not comparable?

 Yes: return NotImplemented.  Note that the == operator automagically
 returns False in this case.

  spam.__eq__(spam)
 True
  spam.__eq__(ham)
 False
  spam.__eq__(23)
 NotImplemented

 This way, the user could explicitly call __eq__ and check for
 NotImplemented if he desires the exceptional behavior.

Thanks, I was unaware of the standard usage of NotImplemented. That
seems like a sensible solution.

URL:http://docs.python.org/ref/types.html#l2h-29

OTOH, that says the truth value is True, so this does not fail:

  assert 'spam'.__eq__(23)
 

Which seems counterintuitive. The explicit value test

  assert 'spam'.__eq__(23) is NotImplemented

would succeed irrespective of truth value,
but it seems contorted and unpythonic to have to write

  assert 'spam'.__eq__(23) is False
 Traceback (most recent call last):
   File stdin, line 1, in ?
 AssertionError
 

to get the effect you might naively expect from

  assert 'spam'.__eq__(23)


I wonder what the rationale for the NotImplemented True truth value was, as 
opposed to False.

On the third hand, I recently posted an idea of using NotImplemented
as a (handy because predefined) logically true sentinel value to return
from a test function to distinguish not_applicable from applicable_and_ok
when desired, yet allowing assert testfun(...) to succeed either way.
That code would have to use another boolishly True sentinel if NotImplemented
were changed to have False truth value.

Maybe it would be useful to have a couple of builtin sentinels like TrueSentinel
and FalseSentinel (or sentinel_t and sentinel_f or pick some names ;-) so we
wouldn't be tempted to misuse unique builtins like NotImplemented and Exception 
etc.

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing text

[Bengt Richter]

On 19 Dec 2005 15:15:10 -0800, sicvic [EMAIL PROTECTED] wrote:

I was wondering if theres a way where python can read through the lines
of a text file searching for a key phrase then writing that line and
all lines following it up to a certain point, such as until it sees a
string of -

Right now I can only have python write just the line the key phrase is
found in.

This sounds like homework, so just a (big) hint: have a look at itertools
dropwhile and takewhile. The solution is potentially a one-liner, depending
on your matching criteria (e.g., case-sensitive fixed string vs regular 
expression).

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do you recommend a good artificial intelligence book?

[Bengt Richter]

On 20 Dec 2005 04:27:29 -0800, Paul Rubin http://[EMAIL PROTECTED] wrote:

Tolga [EMAIL PROTECTED] writes:
 Is there anybody here interested in artificial intelligence (AI)? Yes,
 perhaps this thread would be more suitable for an AI usenet group and I
 have also posted my question there, but I want to know Python
 community's opinion. Could you recommend me please a good AI book to
 start with? I don't want to see Lisp code, if possible ;-D

The classic is Winston and Horn's, which uses Lisp.  You can think of
it as Python with more parentheses if you want.  There's an old book
by Nilsson which is more theoretical.

Have a look at:

http://aima.cs.berkeley.edu/

It's really nicely done.

Note links on above page:


Online Code Repository

* Pseudo-code from the book in pdf or ps.
* Online code in Lisp, Python, Java etc.
* Data for the online code
* Online demos (Java applets and Javascript) 


from the python link (http://aima.cs.berkeley.edu/python/readme.html)


AIMA Python Code
This file gives an overview of the Python code for the
algorithms in the textbook AI: A Modern Approach. The code
is Copyright (c) 2002 by Peter Norvig and is offered free
of charge for your use. As you may know, the textbook presents
algorithms in pseudo-code format; as a supplement we provide
this Python code as well as Lisp code. The intent is to implement
all the algorithms in both languages, so that you can choose whichever
language you prefer. As yet neither implementation is complete,
but the Lisp version is closer.


BTW, Peter Norvig is co-author ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: parsing engineering symbols

[Bengt Richter]

On Tue, 20 Dec 2005 19:07:02 +0530, Suresh Jeevanandam [EMAIL PROTECTED] 
wrote:

Hi,
   I want to convert a string to float value. The string contains 
engineering symbols.
   For example,
   
   s = '12k'

   I want some function which would return 12000
   function(s)
   = 12000.0
   I searched the web, but could not find any function.

It should be easy enough, if you can define precisely what
contains engineering symbols means ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pythonic equivalent of upvar?

[Bengt Richter]

(
   # ... like test above
   )
   # ...
   # use values spelled clv.name or clv['name'] or clv.get('name') etc
   #

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list