Re: Translating pysnmp oids to human readable strings

[rdmurray]

SpamMePlease PleasePlease  wrote:
> On Fri, Mar 6, 2009 at 2:14 PM, Shantanu Joshi  wrote:
> >
> > SpamMePlease PleasePlease  writes:
> >
> >> I actually tried to load the new file with following code:
> >>
> >> print builder.MibBuilder().getMibPath()
> >> mibBuilder = builder.MibBuilder().loadModules('jnx-bgpmib2')
> >>
> >> but I am experiencing the error:
> >>
> >> rivendell # python snmp.py
> >> ('/usr/lib/python2.4/site-packages/pysnmp/v4/smi/mibs/instances',
> >> '/usr/lib/python2.4/site-packages/pysnmp/v4/smi/mibs')
> >> Traceback (most recent call last):
> >>   File "snmp.py", line 7, in ?
> >>     mibBuilder = builder.MibBuilder().loadModules('jnx-bgpmib2')
> >>   File "//usr/lib/python2.4/site-packages/pysnmp/v4/smi/builder.py",
> >> line 82, in loadModules
> >>     raise error.SmiError(
> >> pysnmp.smi.error.SmiError: MIB module
> >> "/usr/lib/python2.4/site-packages/pysnmp/v4/smi/mibs/instances/jnx-bgpmib2.py"
> >> load error: MIB file ".py" not found in search path
> >> rivendell# ls -lA
> >> /usr/lib/python2.4/site-packages/pysnmp/v4/smi/mibs/instances/jnx-bgpmib2.py
> >> -rw-r--r-- 1 root root 2687 Mar  6 06:50
> >> /usr/lib/python2.4/site-packages/pysnmp/v4/smi/mibs/instances/jnx-bgpmib2.py
> >>
> >> Any clue what about is done wrong?
> >
> > The pysnmp library is not able to find your generated file. Either copy
> > your generated file to 
> > /usr/lib/python2.4/site-packages/pysnmp/v4/smi/mibs/instances
> > or call setMibPath on the mibBuilder instance before calling loadModules
> > to add the current directory (or whatever directory holds your generated
> > file).
> > --
> > http://mail.python.org/mailman/listinfo/python-list
> >
> 
> The thing is that I have copied the file (as you may see, I did 'ls
> -lA' on it. If the file wasnt there, the error message was different.
> Also, the setMibPath doesent seem to work, as I tried to set up it to
> any value, and right after that I was performing getMibPath which was
> giving me unchanged default paths. Keep in mind, that whole path setu
> up code has executed without any error.
> So, what is correct and working way to supply my code with that
> converted mib file?

I have absolutely no knowledge of pysnmp, but from reading the error
message I'd guess that you have a '.py' somewhere where you shouldn't.
Either the filename shouldn't have a .py, or you are referring to
the thing to be loaded with a '.py' on the end of the name when you
should be omitting it.

But like I said, I'm speaking out of my hat here, since I've never
worked with pysnmp.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: question about ctrl-d and atexit with threads

[rdmurray]

Darren Dale  wrote:
>On Mar 5, 6:27 pm, "Gabriel Genellina"  wrote:
>> En Thu, 05 Mar 2009 15:26:18 -0200, Darren Dale  
>> escribi�:
>>
>> > On Mar 5, 12:02�pm, s...@pobox.com wrote:
>> >> What happens if you simply call
>>
>> >>my_thread.setDaemon(True)
>>
>> >> (or in Python 2.6):
>>
>> >>my_thread.daemon = True
>>
>> >> ? That is the documented way to exit worker threads when you want the
>> >> application to exit. From the threading module docs:
>>
>> >> "The entire Python program exits when no alive non-daemon threads 
>> >> are left."
>>
>> > Thank you Skip, that solves the problem. I'm still curious what the
>> > difference is between python's handling of sys.exit and EOF, but its
>> > academic at this point.
>>
>> Some applications open a new window for each document they're handling. 
>> When you close the last window, the application exits ("close" does an 
>> implicit "quit"). Note that this does *not* happen when you close the 
>> first, original document you opened, but when there are no more documents 
>> open. The first document is not special in this regard.
>>
>> Python threads work the same way; a thread may finish, but as long as 
>> there are other threads alive, the process continues running. Only after 
>> the last thread has finished, the application quits. The main thread *is* 
>> special sometimes, but not in this aspect,
>> Setting daemon=True is like telling Python "I don't care about this 
>> thread; don't wait for it if that's the only thing you have to do".
>>
>> Calling sys.exit() is an explicit statement: "I want this program to 
>> finish now" (or as soon as possible). It doesn't wait for the remaining 
>> threads (unless you explicitely do so, like in your code).
>
>Right, I understand all that. I don't understand how calling sys.exit
>at the python command line is different from invoking ctrl-D. They
>should both trigger the same mechanism if they are advertised as
>equivalent mechanisms for exiting the interpreter, shouldnt they?

First, just to make sure we are on the same page, I assume you
understand that 'ctlr-D' at the python interpreter prompt is
completely equivalent to your example file that does not call
sys.exit before the end of the script file.  That is, ctrl-D is
"end of file" for the 'script' you are creating at the
interactive interpreter prompt.

When Gabriel says "as long as there are other threads alive, the
process continues running", that is the key to your question.
The main thread has exited (either the interpreter or your main
script file) but another thread is still running (your child
thread), so Python keeps executing that thread.  sys.exit is
_not_ called at the end of the "main" script file, but only after
all threads have exited.

Unless, that is, you call sys.exit explicitly, or set daemon = True.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

RE: Help required to read and print lines based on the type of first character

[rdmurray]

abhinayaraj.r...@emulex.com wrote:
> Thank you for the suggestions.
> 
> Some little reading gave the idea and it works well too. :)
> 
> Here is the code:
> fileIN = open("test.txt")
> count = 0
> for line in fileIN:
>   data= line
> 
>   if '' in data:
>   count = 4
>   elif '###' in data:
>   count = 3
>   elif '##' in data:
>   count = 2
>   elif '#' in data:
>   count = 1
>   elif data.find('#') == -1:
>   count = 0
> 
>   if (count == 0):
>   print data + '\nlooks like a code line...\n'
>   elif(count== 4):
>   print data + '\ninvalid line!\n'
>   elif count>=1:
>   for i in range(0, count):
>   print data

What happens when your input file has a line like:

This is a ## fake line

You might want to check out the strip and startswith methods of strings.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Is it possible to grab hidden code in ClientForm?

[rdmurray]

Muddy Coder  wrote:
> Nowadays some websites let users to fill in some so-called
> verification code, and the tricky thing is that the CODE is delivered
> from server with an image. For example:
> 
>  >  Refresh Image
> 
> When click Refresh Image, the CODE on the image changes. I wonder:
> does the server really send a new image over to browser, or just send
> a hidden code over? Is it possible to parse out such hidden code?
> 
> The other example is:
> 
> 
> 
> 
> I did a few REFRESH on browser, and noticed the CODE changed. But,
> when I viewed the HTML source code, the value "ahZsBmwyLGuf" remained
> the same. What is the trick around here? That is why I am wondering
> whether the server really keeps sending new images over, or just
> sending new hidden codes over. If the trick is just keep sending
> hidden codes over, there might be a chance of capturing the codes. Can
> somebody help me out? Thanks a lot!

If there were some way for a program to extract the code from what
the server sent, then the whole point of using the Captcha (preventing
spam robots from posting) would be defeated.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Creating Zip file like java jar file

[rdmurray]

zaheer.ag...@gmail.com wrote:
> On Feb 28, 11:33 pm, Lie Ryan  wrote:
> > zaheer.ag...@gmail.com wrote:
> > > On Feb 28, 11:15 pm, "Gabriel Genellina" 
> > > wrote:
> > >> En Sat, 28 Feb 2009 14:34:15 -0200,  escribi=
> =F3:
> >
> > >>> I want to create zip file equivalent to java jar file,I created a zip
> > >>> file of my sources and added some __main__.py
> > >>> it says __Main__.py not found in Copyproject.zip..?
> > >> __main__.py must exist in the root directory.
> >
> > >> --
> > >> Gabriel Genellina
> >
> > > What do you mean by root directory..?What is this directory in
> > > Windows..?You mean the top level folder of the project?in my case
> > > copyproject folder..?
> >
> > root directory is the topmost directory (imagine a tree, the root is
> > where all the branches branched from), in this case the root directory
> > is your copyproject main folder.
> 
> 
> I wonder, I do have the __main__.py  in the root directory,but still getting
> python.exe cannot find __main__.py in CopyProject.zip
> 
> I used the following command
> python Copyproject.zip -m __main__.py --uploadFile and also tried with
> python Copyproject.zip --uploadFile
> both give me the same error , I can see the sys.path contains the
> project folder.

How did you create the zip?  The __main__.py file has to be at the
_top level_ of the zip file.  In other words:

zip test.zip __main.py othermodule.py somedir

works but

zip test.zip myproject

does not.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: How best to test functions which use date.today

[rdmurray]

Christian Heimes  wrote:
> Lie Ryan wrote:
> >> But this fails with:
> >>
> >> TypeError: can't set attributes of built-in/extension type
> >> 'datetime.date'
> > 
> > This is because today is an attribute. In python, we can override
> > attribute access to become a function call. I don't have python right
> > now, but try this:
> > 
> > del date.today
> > date.today = mytoday
> 
> It won't work. The datetime module is written in C. You can't modify a C
>  extension.

Hmm.  Given that, Lie, maybe what you need to do is modify your code
so that you call your own special purpose function to get 'today',
and replace _that_ for testing, using datetime's today for production.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

OTish: convince the team to drop VBScript

[rdmurray]

"Christian R."  wrote:
> The company does use Python on rare occasions. It all comes down to
> the prejudices and habits of one of the programmers. His only argument
> I can't counter -because I don't see the problem- is that "Python
> modules cause problems for updates to customer's installations".

You don't really need arguments so much as you need negotiation, people
management and (ugh) political skills, I think.  Your task is to overcome
his prejudices, and you aren't going to do that with arguments, no matter
how good.

IMO the first thing you ought to do is dig in, really listen, and find
out what his issue is with module distribution.

Listening well is your most powerful asset.  Overcome your own prejudices
first, and his may follow :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

TypeErrors

[rdmurray]

Sean Novick  wrote:
> First lookup:
> Traceback (most recent call last):
>File "F:\CSC113 Module 4 CA\sample database.py", line 72, in 
>class phonedb:
>File "F:\CSC113 Module 4 CA\sample database.py", line 146, in phonedb
>for entry in foo.lookup('reifsch'):
>File "F:\CSC113 Module 4 CA\sample database.py", line 101, in lookup
>if cmp(e, string) =3D=3D 0:
> TypeError: comparison did not return an int
>
> I do not know what this error means. I tried to look it up in help, but to no
> avail. If someone could please help or tell me where I could find an answer.
> Thanks.

The 'cmp' function (which is depricated, by the way) asks the two
arguments to compare themselves.  It does this by invoking the __cmp__
method on the first object, passing it the second.  (Actually it's more
complicated than that, but it will do as a partial explanation for now).
The __cmp__ method is expected to return an integer, which represents
the results of doing the comparison (see the docs if you want to know
the values...but as I said cmp and __cmp__ a depricated).

So, whatever 'e' is (and we can't tell from the traceback, you'll have
to work through your code to figure it out, probably by sprinkling in
'print' statements), its __cmp__ method didn't return an integer.

When you fix this, I would recommend converting to using rich comparisons
(__eq__, __lt__), etc, since __cmp__ is eventually going away (it doesn't
exist in Python 3.x).

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: why cannot assign to function call

[rdmurray]

Mark Wooding  wrote:
> Ethan Furman  writes:
> 
> > Mark Wooding wrote:
> >> Here's what I think is the defining property of pass-by-value [...]:
> >>
> >>   The callee's parameters are /new variables/, initialized /as if by
> >>   assignment/ from the values of caller's argument expressions.
> >>
> >> My soundbite definition for pass-by-reference is this:
> >>
> >>   The callee's parameters are merely /new names/ for the caller's
> >>   argument variables -- as far as that makes sense.
> >
> > Greetings, Mark!
> 
> You posted to the whole group -- probably using a wide-reply option
> while reading the mailing list.  Still, I'll give an answer here in
> order to help any other readers of the list or newsgroup.  However, as
> far as I'm concerned, this discussion is basically at an end and I'm not
> really interested in starting it up all over again.  To that end, I've
> set followups to `poster'.  I shall continue reply to private email
> which seems interested in a sensible discussion.

I just popped in to this thread, and read the whole thing in a marathon
session (why, I'm not quite sure, but somehow I found it interesting :).
I'm going to follow up here at the risk of annoying Mark, because I
think there may be a way to reconcile the language in a way that is
helpful in explaining things to Python beginners.

> > I was hoping you might be able to clarify those last two sound bites
> > for me -- I think I understand what you are saying, but I'm confused
> > about how they relate to Python...

I think this is the key point.  I am an experienced Python programmer,
and I had to think long and hard about what you were saying, Mark, in
order to understand how it applied to Python.  I think this means that
your model and/or your level of abstraction is not "natural" when trying
to understand Python programs, and I think I may have figured out why.

> > Specifically, how is a new name (pbr) different, in Python, from a new
> > name initialized as if by assignment (pbv)?  It seems to me than you
> > end up with the same thing in either case (in Python, at least),
> > making the distinction non-existent.
> 
> You've missed a level of indirection.  In particular, `names' aren't
> things that you initialize.  They're things that you /bind/ to
> variables.  The crucial difference is that, in pass-by-value, new
> variables are created as an intrinsic part of the process, whereas in
> pass-by-reference, new variables are not (usually) created, and instead
> the formal parameter names are bound to the caller's pre-existing
> argument variables.

I think that the reason Ethan missed that level of indirection is that
Python hides that level of indirection from the programmer.  And I think
that this is where the conversation between you and Steven got hung
up, Mark.  You are firmly grounded in how languages work in general,
so your conceptual model naturally includes details that Steven's
conceptual model can ignore, because he's dealing only with the model
used by Python.  He can (and wants to!) ignore the level of indirection
that Python doesn't even give him access to.

In short, there is a reason why you almost never hear the word 'variable'
from an experienced Python programmer when talking about the Python model.
That level detail is abstracted into something else by the Python
conceptual model: it becomes a namespace mapping names to objects.

> It's worth noting that I use the terms `name' and `binding' in different
> ways from most of the Python community.  This is unfortunate.  The
> discrepancy is actually because the Python meanings of these words are
> not the same as the meanings in the wider computer science and
> mathematical communities.  For example, many Python users seem to use
> `binding' to mean `assignment', which is a shame because it leaves the
> concept that is usually called `binding' without a name.  So I'll stick
> with the wider meanings.

It seems to me from reading the thread that everyone is actually in
pretty good agreement that 'name' is the symbol one types in the
program source to denote...something.  It's the something where things
get tricky.  So let's leave name itself alone.

But let's replace 'assignment' and 'binding' with the concept that is
more natural to Python's model, that of a "mapping".  A namespace maps
names to objects.  If I'm understanding you correctly, Mark, in your
model what I'm calling a namespace is the set of variable bindings in a
particular environment.  Because Python does not allow the programmer
to actually rebind a variable (Python handles the association between
name and variable completely behind the scenes), Python programmers
naturally skip past that name-to-variable binding step, and conceptually
think about the name being bound to the _value_ instead. After all,
that's the only thing they can change.  This may be unfortunate from
a precisionist viewpoint, but it is both natural and, IMO, inevitable,
because it makes thinking about Python programs m

Efficient searching through objects

[rdmurray]

sert  wrote:
> I have written a program that reads data and updates the records 
> for some people. They are represented by objects, and I need to 
> read the data from a file, look the person up and then update 
> his record.
> 
> I have implemented this by creating a list with all the people's 
> names and another list with their objects (their data).
> 
> It works but after profiling the code it turns out that half the 
> time spent in the program is spent in the list.index() function 
> looking up names. Isn't there a less goofy and more efficient 
> way of doing this?

It sounds like what you are looking for is the dictionary
data type.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I decode unicode characters in the subject using email.message_from_string()?

[rdmurray]

Steve Holden  wrote:
> rdmur...@bitdance.com wrote:
> > Steve Holden  wrote:
> > from email.header import decode_header
> > print
> >> decode_header("=?us-ascii?Q?Inteum_C/SR_User_Tip:__Quick_Access_to_Recently_Opened_Inteu?=\r\n\t=?us-ascii?Q?m_C/SR_Records?=")
> >> [('Inteum C/SR User Tip:  Quick Access to Recently Opened Inteum C/SR
> >> Records', 'us-ascii')]
> > 
> > It is interesting that decode_header does what I would consider to be
> > the right thing (from a pragmatic standpoint) with that particular bit
> > of Microsoft not-quite-standards-compliant brain-damage; but, removing
> > the tab is not in fact standards compliant if I'm reading the RFC
> > correctly.
> > 
> You'd need to quote me chapter and verse on that. I understood that the
> tab simply indicated continuation, but it's a *long* time since I read
> the RFCs.

Tab is not mentioned in RFC 2822 except to say that it is a valid
whitespace character.  Header folding (insertion of ) can
occur most places whitespace appears, and is defined in section
2.2.3 thusly:

   Each header field is logically a single line of characters comprising
   the field name, the colon, and the field body.  For convenience
   however, and to deal with the 998/78 character limitations per line,
   the field body portion of a header field can be split into a multiple
   line representation; this is called "folding".  The general rule is
   that wherever this standard allows for folding white space (not
   simply WSP characters), a CRLF may be inserted before any WSP.  For
   example, the header field:

   Subject: This is a test

   can be represented as:

   Subject: This
is a test

   [irrelevant note elided]

   The process of moving from this folded multiple-line representation
   of a header field to its single line representation is called
   "unfolding". Unfolding is accomplished by simply removing any CRLF
   that is immediately followed by WSP.  Each header field should be
   treated in its unfolded form for further syntactic and semantic
   evaluation.

So, the whitespace characters are supposed to be left unchanged
after unfolding.

--David

--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I decode unicode characters in the subject using email.message_from_string()?

[rdmurray]

Steve Holden  wrote:
> >>> from email.header import decode_header
> >>> print
> decode_header("=?us-ascii?Q?Inteum_C/SR_User_Tip:__Quick_Access_to_Recently_Opened_Inteu?=\r\n\t=?us-ascii?Q?m_C/SR_Records?=")
> [('Inteum C/SR User Tip:  Quick Access to Recently Opened Inteum C/SR
> Records', 'us-ascii')]
> >>>

It is interesting that decode_header does what I would consider to be
the right thing (from a pragmatic standpoint) with that particular bit
of Microsoft not-quite-standards-compliant brain-damage; but, removing
the tab is not in fact standards compliant if I'm reading the RFC
correctly.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I decode unicode characters in the subject using email.message_from_string()?

[rdmurray]

John Machin  wrote:
> On Feb 25, 11:07=A0am, "Roy H. Han" 
> wrote:
> > Dear python-list,
> >
> > I'm having some trouble decoding an email header using the standard
> > imaplib.IMAP4 class and email.message_from_string method.
> >
> > In particular, email.message_from_string() does not seem to properly
> > decode unicode characters in the subject.
> >
> > How do I decode unicode characters in the subject?
> 
> You don't. You can't. You decode str objects into unicode objects. You
> encode unicode objects into str objects. If your input is not a str
> object, you have a problem.

I can't speak for the OP, but I had a similar (and possibly
identical-in-intent) question.  Suppose you have a Subject line that
looks like this:

Subject: 'u' Obselete type =?ISO-8859-1?Q?--_it_is_identical_?=   
=?ISO-8859-1?Q?to_=27d=27=2E_=287=29?=

How do you get the email module to decode that into unicode?  The same
question applies to the other header lines, and the answer is it isn't
easy, and I had to read and reread the docs and experiment for a while
to figure it out.  I understand there's going to be a sprint on the
email module at pycon, maybe some of this will get improved then.

Here's the final version of my test program.  The third to last line is
one I thought ought to work given that Header has a __unicode__ method.
The final line is the one that did work (note the kludge to turn None
into 'ascii'...IMO 'ascii' is what deocde_header _should_ be returning,
and this code shows why!)

---
from email import message_from_string
from email.header import Header, decode_header

x = message_from_string("""\
To: test
Subject: 'u' Obselete type =?ISO-8859-1?Q?--_it_is_identical_?=   
=?ISO-8859-1?Q?to_=27d=27=2E_=287=29?=

this is a test.
""")

print x
print ""
for key, header in x.items():
print key, 'type', type(header)
print key+":", unicode(Header(header)).decode('utf-8')
print key+":", decode_header(header)
print key+":", ''.join([s.decode(t or 'ascii') for (s, t) in 
decode_header(header)]).encode('utf-8')
---


From nobody Wed Feb 25 08:35:29 2009
To: test
Subject: 'u' Obselete type =?ISO-8859-1?Q?--_it_is_identical_?=
=?ISO-8859-1?Q?to_=27d=27=2E_=287=29?=

this is a test.


To type 
To: test
To: [('test', None)]
To: test
Subject type 
Subject: 'u' Obselete type =?ISO-8859-1?Q?--_it_is_identical_?=   
=?ISO-8859-1?Q?to_=27d=27=2E_=287=29?=
Subject: [("'u' Obselete type", None), ("-- it is identical to 'd'. (7)", 
'iso-8859-1')]
Subject: 'u' Obselete type-- it is identical to 'd'. (7)


--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: can error messages be improved or can they be overridden ?

[rdmurray]

"andrew cooke"  wrote:
> rdmur...@bitdance.com wrote:
> [...]
> > (You know, I really ought to revisit that routine and make it part
> > of my standard development toolbox.)
> 
> please post it

OK.  I dug it up, cut out the stuff that was specific to the application,
freshened it up a little, and added a little demo.  I don't claim this
is the best possible way to do this (it certainly could be made smarter,
possibly by using python facilities to actually parse the line; right
now it takes a very brute force approach to trying to find variables),
but it served my purposes and perhaps it will help others.

Oh, yeah, and I just added the globals bit without thoroughly testing
it...I think it probably doesn't help much and may be misleading when
the traceback is thrown in an imported module.

Enjoy.

--RDM

--
from traceback import format_tb, format_exception_only
from sys import exc_info
from re import compile
varsplitter = compile("[^0-9a-zA-Z_]")

def format_exception():
"""
Add a dump of any variables we can identify from the failing program
line to the end of the traceback.  The deep mojo for doing this came
from an example in the Zope core plus documentation in the Python
Quick Reference.
"""
etype, value, tb = exc_info()
plaintb = format_tb(tb)
result=['Traceback (innermost last):']
for line in plaintb:
result.append(line)
f = tb.tb_frame
tb = tb.tb_next
locals=f.f_locals
vars = varsplitter.split(line.split('\n')[-2])
dvars = set()
self = None
if 'self' in locals: self = locals['self']
for v in vars:
if v in dvars: continue
dvars.add(v)
if v in locals:
result.append('  %s: %r\n' % (v,locals[v]))
if self and hasattr(self, v):
result.append('  self.%s: %r\n' % (v,getattr(self, v)))
if v in globals():
result.append('  (global) %s: %r\n' % (v,globals()[v]))
result.extend(format_exception_only(etype, value))
return ''.join(result)


class Demo:
y = 200
def __init__(self, x):
self.x = x

def bad(self):
x = [1, 2]
y = (5, 9, 9)
y[1] = x[2] + foo

foo = 'a value'

def main():
bar = Demo('some value')
bar.bad()

  
if __name__=='__main__':
try: main()
except Exception:
print format_exception()

--
rdmur...@maestro:~>python tb.py
Traceback (innermost last):  File "tb.py", line 56, in 
try: main()
  File "tb.py", line 52, in main
bar.bad()
  File "tb.py", line 46, in bad
y[1] = x[2] + foo
  y: (5, 9, 9)
  self.y: 200
  x: [1, 2]
  self.x: 'some value'
  (global) foo: 'a value'
IndexError: list index out of range
--

--
http://mail.python.org/mailman/listinfo/python-list

read csv error question

[rdmurray]

Vincent Davis  wrote:
> I am trying to read a csv file from excel on a mac. I get the following
> error.SystemExit: file some.csv, line 1: new-line character seen in unquoted
> field - do you need to open the file in universal-newline mode?
> I was using the example code
> import csv, sys
> 
> reader = csv.reader(open('/Volumes/vincentdavis
> 2/match/data/matchdata2008.csv', "rb"))
> try:
> for row in reader:
> print row
> except csv.Error, e:
> sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e))
> 
> I think this has to do with the end of line character but I am unsure how to
> fix it. I don't what to change the actual csv file I would like to fix the
> code.

You could try reading the error message and looking at the documentation
of the 'open' function.  Pay particular attention to the keywords
'universal-newline mode'.

Hint: 'rb' is almost the opposite of universal newline mode, and
it is very rare that you'd want to use 'b' to read a text file.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: can error messages be improved or can they be overridden ?

[rdmurray]

Stef Mientki  wrote:
> thanks Ron,
> 
> but I was looking for a more general solution,
> in which I don't change the program itself,
> and where the error messages (in general) become more informative than 
> it is by default.
[snip]
> >> -Original Message-
> >> From: Stef Mientki [mailto:stef.mien...@gmail.com] 
[snip]
> >> I often get an error message like this
> >>
> >> self.Brick.Par [ self.EP[2] ]['FileName'] = filename
> >> IndexError: list index out of range
> >>
> >> Now it would be very welcome,
> >> if the error message specified which index is out of range, 
> >> in this case e.g.,
> >> - specifying the length of self.EP
> >> - specifying the value of self.EP[2]
> >> - specifying the length of self.Brick.Par etc..
> >>
> >> Is there a way to override the error message and provide this 
> >> information, or is it impossible ?
> >>
> >> And if it's possible, am I the only one who often stumbles 
> >> about this problem ?
> >> (I expect many people must have bounced in this problem before me ;-)

FYI, top posts are much harder to read and to reply to than if you edit
the message to which you are replying and add your new content interleaved
or at the bottom (as I have done).

No you are not the only one who wishes the error messages were
more informative.  In one complex application I had, I made my life
easier with a hack I copied from Zope.  Briefly, at the top level
of the program I trap all exceptions, get the traceback object from
sys.exc_info, format it with format_tb, and then process it to add info.
I applied several enhancements, but the one relevant to you was to grab
the locals dictionary from the last frame of the traceback, and use a
regex to split the last source line in the formatted traceback up into
candidate variable names.  Then I printed the name and the repr of the
value of any of those names I found in the locals dict.

That wouldn't tell you directly which variable it was threw the index
error, but having the current values of the variables involved saves a
lot of time (no need to go back and stick in prints in many cases).

(You know, I really ought to revisit that routine and make it part
of my standard development toolbox.)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Pythonic way to determine if one char of many in a string

[rdmurray]

odeits  wrote:
> On Feb 21, 12:47=A0am, "Gabriel Genellina" 
> wrote:
> > En Sat, 21 Feb 2009 01:14:02 -0200, odeits  escribi=F3:
> >
> > > On Feb 15, 11:31=A0pm, odeits  wrote:
> > >> It seems what you are actually testing for is if the intersection of
> > >> the two sets is not empty where the first set is the characters in
> > >> your word and the second set is the characters in your defined string.
> >
> > > To expand on what I was saying I thought i should provide a code
> > > snippet:
> >
> > > WORD = 'g' * 100
> > > WORD2 = 'g' * 50 + 'U'
> > > VOWELS = 'aeiouAEIOU'
> > > BIGWORD = 'g' * 1 + 'U'
> >
> > > def set_test(vowels, word):
> >
> > >  vowels = set( iter(vowels))
> > >  letters = set( iter(word) )
> >
> > >  if letters & vowels:
> > >  return True
> > >  else:
> > > return False
> >
> > > with python 2.5 I got 1.30 usec/pass against the BIGWORD
> >
> > You could make it slightly faster by removing the iter() call:
> > letters = set(word)
> > And (if vowels are really constant) you could pre-build the vowels set.
> 
> set(word) = set{[word]} meaning a set with one element, the string
> the call to iter makes it set of the letters making up the word.

Did you try it?

Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> set('abcd')
set(['a', 'c', 'b', 'd'])

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Find the location of a loaded module

[rdmurray]

"Gabriel Genellina"  wrote:
> En Fri, 20 Feb 2009 20:44:21 -0200, Aaron Scott
>  escribi=F3:
> 
> > So, the problem lies with how Python cached the modules in memory.
> > Yes, the modules were in two different locations and yes, the one that
> > I specified using its direct path should be the one loaded. The
> > problem is, the module isn't always loaded -- if it's already in
> > memory, it'll use that instead. And since the modules had the same
> > name, Python wouldn't distinguish between them, even though they
> > weren't exactly the same.
> 
> Yes, that's how import works. It's barely documented, and you finally
> learned it the hard way...

I'd argue a little bit with "barely documented".  In the reference, the
discussion of the import statement starts off saying:

Import statements are executed in two steps: (1) find a module,
and initialize it if necessary; (2) define a name or names in the
local namespace (of the scope where the import statement occurs).

The third paragraph then says:

The system maintains a table of modules that have been or are being
initialized, indexed by module name. This table is accessible as
sys.modules. When a module name is found in this table, step (1)
is finished.

That is pretty up front and unambiguous documentation.

However, the consequences of that statement won't be immediately clear
on first reading.  I think it would have cleared up Aaron's confusion
if he'd happened to think to read it.  But since he knew the syntax
of the import statement already, I'm not surprised he did not read it.

The Tutorial, in the section on modules and import, says:

A module can contain executable statements as well as function
definitions. These statements are intended to initialize the
module. They are executed only the first time the module is imported
somewhere.

This is considerably less precise, if more superficially understandable.
I wonder if it would be worth expanding on that statement to mention
that the module is not even looked for on disk if a module by that
name has already been imported.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: "Maximum recursion depth exceeded"...why?

[rdmurray]

Thomas Allen  wrote:
> On Feb 18, 4:51 am, alex23  wrote:
> > On Feb 18, 7:34 pm, rdmur...@bitdance.com wrote:
> >
> > > Yeah, but wget -r -k will do that bit of it, too.
> >
> > Wow, nice, I don't know why I never noticed that. Cheers!
> 
> Hm...doesn't do that over here. I thought it may have been because of
> absolute links (not to site root), but it even leaves things like  href="/">. Does it work for you guys?

It works for me.  The sample pages I just tested on it don't use
any href="/" links, but my 'href="/about.html"' got properly
converted to 'href="../about.html"'.  (On the other hand my '/contact.html'
got converted to a full external URL...but that's apparently because the
contact.html file doesn't actually exist :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: "Maximum recursion depth exceeded"...why?

[rdmurray]

alex23  wrote:
> On Feb 18, 11:36=A0am, Paul Rubin  wrote:
> > Thomas Allen  writes:
> > > attempting. Basically, I'm transforming a live site to a local one and
> >
> > Something wrong with wget -R ?
> 
> Did you mean wget -r ?
> 
> That will just grab the entire site, though. I'm guessing that Thomas'
> function absToRel will eventually replace the print with something
> that changes links accordingly so the local version is traversable.

Yeah, but wget -r -k will do that bit of it, too.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: hpw to convert a linux python script ?

[rdmurray]

Stef Mientki  wrote:
> Alec Schueler wrote:
> > On Feb 11, 7:58 pm, Stef Mientki  wrote:
> >   
> >> As there are a whole lot of these lines, in a whole lot of files,
> >> I wonder if there's a simple trick to point
> >>   /usr/share/tinybldLin/
> >> to my directory ?
> >>
> >> thanks,
> >> Stef
> >> 
> >
> > Find and replace?
> >   
> well I was thinking of a more elegant way,
> so the scripts remains, or better become multi-platform.
> 
> anyway thanks,

Well, you'll have to rewrite the script so that it doesn't make
assumptions about path names, and we can't help you with that without
seeing some of the script code.  There _might_ be easy fixes, but if the
path is hardcoded in many places in the script...then you will just have
to go through and un-hard-code it.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: what IDE is the best to write python?

[rdmurray]

a...@pythoncraft.com (Aahz) wrote:
> In article ,
>   wrote:
> >Quoth a...@pythoncraft.com (Aahz):
> >> Then I have the problem of copying around the syntax highlighting
> >> configuration to every computer I use.
> >
> >Well, _that's_ easy to fix.  I have a little bash script called
> >'synchome' that uses rsync to update the home directory on any of the
> >remote machines on which I work.  I've had to install rsync on one or
> >two of the boxes, but that's a useful thing to do anyway.
> >
> >(Granted, I still have a couple bugs to work out, where I haven't taken
> >the time to conditionalize things properly for some of the more exotic
> >machine configurations, but hey, if I spend more time on those machines
> >I'll get around to it...)
> 
> The need to conditionalize for different environments is the main reason
> I haven't bothered, combined with the need to spend time working out a
> decent color scheme I'm happy with.  I've been working this way (plain
> vi) for more than twenty years and I don't see much point fixing what
> ain't b0rken.

I did the conditionalization bit by bit, and there isn't much of it
since I can install my "standard environment" (zsh, screen, vim) on
almost all the boxes on which I work.

I was as you are on colorization until about three months ago.  Then I
figured out how to get 256 colors in my xterms and found 'desert.vim',
which works for me (nice gentle colors).  Now I'm a happy user of
colorization, and have even written a colorizor for a file whose syntax
I defined, that helps me avoid making stupid typos in said file, that
I used to make far too often.  In other words, the benefits tipped over
into outweighing the costs for me just recently...

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Can urllib check path exists on server?

[rdmurray]

Muddy Coder  wrote:
> urllib bridges up a client to a server, it works fine. I wonder: is
> there a method that can check the existence of a file in the server
> side? We can check such an existence on local filesystem by using
> os.path.exists(), can I do such a check on server? For example,
> http://somedomain.com/foo/bar.jpg is residing in a hosting server, can
> I use urllib to check if bar.jpg file existing or not? Thanks!

urllib implements the http protocol for http URLs, so you can only do
what the http protocol allows.  Thus if http://somedomain.com/foo/bar.jpg
is a valid URL on that server, you can do a urlopen, and then check
the 'getcode' method on the returned object to see if you got a '404'
(not found) code back.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Import without executing module

[rdmurray]

Lie Ryan  wrote:
> On Tue, 03 Feb 2009 20:08:34 -0800, Stephen Hansen wrote:
> 
> > There is no need to try to make sure something is
> > executed/compiled only once in Python like you may want to do in C.
> > Every module is only ever compiled once: if you import it ten times in
> > ten different places only the first will compile and execute the code,
> > the rest of the calls will return that original module again.
> 
> 
> If you import a module, it will be recompiled only if the module changes 
> from the precompiled .pyc/.pyo. OTOH, if you run a module, it will be 
> recompiled every time even if there is no change.
> 

As long as we are nitpicking, and trying to clarify things for a new
Python userlet me give it a try.  By doing this I get to find out
if I really understand it as well as I think I do :)

If a file is run as a script ('python somefile.py', or shebang magic on
unix or the equivalent on other OSes), then the code is compiled/executed
each time that is done.  If a module is imported (or something is imported
from a module), then the _first time_ during a given run of the python
interpreter that such an import is done, the timestamp of the .py file
(if it exists) is compared to the timestamp of the .pyc/.pyo file, and
if the .py is newer, the .py file is compiled to byte code, the resulting
byte code is written to the .pyc (or .pyo), and the byte code is executed.
If the .py file is older, then the saved byte code is loaded and executed.
It is the execution of the byte code that creates the module's namespace
(a collection of names pointing to objects).  This namespace is registered
in sys.modules under the module's name.  During the remainder of that
run of the python interpreter, any import statement that draws from
that module simply loads a pointer to the module's namespace (or to
objects referenced by the module's namespace if it is a 'import from')
into the local namespace.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Help needed to retrieve text from a text-file using RegEx

[rdmurray]

Oltmans  wrote:
> Here is the scenario:
> 
> It's a command line program. I ask user for a input string. Based on
> that input string I retrieve text from a text file. My text file looks
> like following
> 
> Text-file:
> -
> AbcManager=C:\source\code\Modules\Code-AbcManager\
> AbcTest=C:\source\code\Modules\Code-AbcTest\
> DecConnector=C:\source\code\Modules\Code-DecConnector\
> GHIManager=C:\source\code\Modules\Code-GHIManager\
> JKLConnector=C:\source\code\Modules\Code-JKLConnector
> 
> -
> 
> So now if I run the program and user enters
> 
> DecConnector
> 
> Then I'm supposed to show them this text "C:\source\code\Modules\Code-
> DecConnector" from the text-file. Right now I'm retrieving using the
> following code which seems quite ineffecient and inelegant at the same
> time
> 
>  with open('MyTextFile.txt') as file:
>  for line in file:
>  if mName in line: #mName is the string that contains user input
>  Path =str(line).strip('\n')
>  tempStr=Path
>  Path=tempStr.replace(mName+'=',"",1)

I've normalized your indentation and spacing, for clarity.

> I was wondering if using RegEx will make this look better. If so, can
> you please suggest a Regular Expression for this? Any help is highly
> appreciated. Thank you.

This smells like it might be homework, but I'm hoping you'll learn some
useful python from what follows regardless of whether it is or not.

Since your complaint is that the above code is inelegant and inefficient,
let's clean it up.  The first three lines that open the file and set up
your loop are good, and I think you will agree that they are pretty clean.
So, I'm just going to help you clean up the loop body.

'line' is already a string, since it was read from a file.  No need to
wrap it in 'str':

  Path = line.strip('\n')
  tempStr=Path
  Path=tempStr.replace(mName+'=',"",1)

'strip' removes characters from _both_ ends of the string.  If you are
trying to make sure that you _only_ strip a trailing newline, then you
should be using rstrip.  If, on the other hand, you just want to get
rid of any leading or trailing whitespace, you could just call 'strip()'.
Since your goal is to print the text from after the '=', I'll assume
that stripping whitespace is desirable:

  Path = line.strip()
  tempStr=Path
  Path=tempStr.replace(mName+'=',"",1)

The statement 'tempStr=Path' doesn't do what you think it does.
It just creates an alternate name for the string pointed to by Path.
Further, there is no need to have an intermediate variable to hold a
value during transformation.  The right hand side is computed, using
the current values of any variables mentioned, and _then_ the left hand
side is rebound to point to the result of the computation.  So we can
just drop that line entirely, and use 'Path' in the 'replace' statement:

  Path = line.strip()
  Path = Path.replace(mName+'=',"",1)

However, you can also chain method calls, so really there's no need for
two statements here, since both calls are simple:

  Path = line.strip().replace(mName+'=',"",1)

To make things even simpler, Python has a 'split' function.  Given the
syntax of your input file I think we can assume that '=' never appears
in a variable name.  split returns a list of strings constructed by
breaking the input string at the split character, and it has an optional
argument that gives the maximum number of splits to make.  So by doing
'split('=', 1), we will get back a list consisting of the variable name
and the remainder of the line.  The remainder of the line is exactly
what you are looking for, and that will be the second element of the
returned list.  So now your loop body is:

  Path = line.strip().split('=', 1)[1]

and your whole loop looks like this:

with open('MyTextFile.txt') as file:
for line in file:
if mName in line:
Path = line.strip().split('=', 1)[1]

I think that looks pretty elegant.  Oh, and you might want to add a
'break' statement to the loop, and also an 'else:' clause (to the for
loop) so you can issue a 'not found' message to the user if they type
in a name that does not appear in the input file.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: isfifo?

[rdmurray]

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=  wrote:
> rdmur...@bitdance.com wrote:
> > I've googled and looked through os.path, but I don't see a method for
> > determining if a path points to a FIFO.  Anyone know of a simple way to
> > do so?
> 
> def isfifo(fn):
>   return stat.S_ISFIFO(os.stat(fn).st_mode)

Thanks.  I should have thought of looking in os.stat.  Odd that S_ISFIFO
didn't come up in my google search.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

isfifo?

[rdmurray]

I've googled and looked through os.path, but I don't see a method for
determining if a path points to a FIFO.  Anyone know of a simple way to
do so?

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Rhamphoryncus  wrote:
> On Feb 6, 10:21=A0pm, rdmur...@bitdance.com wrote:
> > Quoth Mensanator :
> > > def flatten(listOfLists):
> > > =A0 =A0 return list(chain.from_iterable(listOfLists))
> >
> > =A0 =A0 Python 2.6.1 (r261:67515, Jan =A07 2009, 17:09:13)
> > =A0 =A0 [GCC 4.3.2] on linux2
> > =A0 =A0 Type "help", "copyright", "credits" or "license" for more informa=
> tion.
> > =A0 =A0 >>> from itertools import chain
> > =A0 =A0 >>> list(chain.from_iterable([1, 2, [3, 4]]))
> > =A0 =A0 Traceback (most recent call last):
> > =A0 =A0 =A0 File "", line 1, in 
> > =A0 =A0 TypeError: 'int' object is not iterable
> > =A0 =A0 >>> list(chain(*[1, 2, [3, 4]]))
> > =A0 =A0 Traceback (most recent call last):
> > =A0 =A0 =A0 File "", line 1, in 
> > =A0 =A0 TypeError: 'int' object is not iterable
> > =A0 =A0 >>> list(chain.from_iterable(['abcd', 'efg', [3, 4]]))
> > =A0 =A0 ['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4]
> 
> What usecase do you have for such inconsistently structured data?
> 
> If I'm building a tree I use my own type for the nodes, keeping them
> purely internal, so I can always use isinstance without worrying about
> getting something inconvenient passed in.

I don't have any use cases myself, I'm just pointing out that this
doesn't answer the concerns of the OP, who presumably does.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

urllib2: problem of handling space in parameter

[rdmurray]

Quoth Muddy Coder :
> Hi Folks,
> 
> I encrountered a problem of using urllib2: the space handling. Look at
> the code below:
> 
> import urllib2
> url = r'http://somedomain.com/a.cgi?name=muddy coder&password=foobar
> cgi_back = urllib2.urlopen(url).read()
> 
> In this cgi_back, I saw field password worked fine, but field name
> not, only muddy was picked up by CGI. So, I had to cover the space, by
> using syntax name=muddy-coder, it went through. So, I presume urllib2
> may have an approach of handling white space in this regard. Can
> anybody help? Thanks!

urllib.urlencode.  Unecoded spaces aren't actually valid in a URL.

--David

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Quoth Mensanator :
> On Feb 6, 3:23=A0pm, Rhamphoryncus  wrote:
> > On Feb 5, 1:16=A0pm, Michele Simionato 
> > wrote:
> >
> > > On Feb 5, 7:24=A0pm, a...@pythoncraft.com (Aahz) wrote:
> > > > In article 
> > > > ,
> > > > Michele Simionato =A0 wrote:
> > > > >Looks fine to me. In some situations you may also use hasattr(el,
> > > > >'__iter__') instead of isinstance(el, list) (it depends if you want to
> > > > >flatten generic iterables or only lists).
> > > > Of course, once you do that, you need to special-case strings...
> >
> > > Strings are iterable but have no __iter__ method, which is fine in
> > > this context, since I would say 99.9% of times one wants to treat them
> > > as atomic objects, so no need to special case.
> >
> > Don't worry, that little oddity was fixed for you:
> >
> > Python 3.0+ (unknown, Dec =A08 2008, 14:26:15)
> > [GCC 4.3.2] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>>  str.__iter__
> >  > >>> bytes.__iter__
> >  > >>> bytearray.__iter__
> > 
> >
> > I'm in the "why do you need more than 1 depth?" camp. Dispatching
> > based on your own type should be given an extra look. Dispatching
> > based passed in types should be given three extra looks.
> >
> > I didn't realize itertools.chain(*iterable) worked. I guess that
> > needs to be pushed as the canonical form.
> 
> What about this (from the Recipes section of the itertools manual)?
> 
> def flatten(listOfLists):
> return list(chain.from_iterable(listOfLists))

Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from itertools import chain
>>> list(chain.from_iterable([1, 2, [3, 4]]))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'int' object is not iterable
>>> list(chain(*[1, 2, [3, 4]]))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'int' object is not iterable
>>> list(chain.from_iterable(['abcd', 'efg', [3, 4]]))
['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4]

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Trouble sorting a list of objects by attributes

[rdmurray]

Quoth Robocop :
> Hello again,
> I've found myself stumped when trying to organize this list of
> objects.  The objects in question are timesheets which i'd like to
> sort by four attributes:
> 
> class TimeSheet:
>   department = string
>   engagement = string
>   date = datetime.date
>   stare_hour = datetime.time
> 
> My ultimate goal is to have a list of this timesheet objects which are
> first sorted by departments, then within each department block of the
> list, have it organized by projects.  Within each project block i
> finally want them sorted chronologically by date and time.
> 
> To sort the strings i tried:
> 
> timesheets.sort(key=operator.attrgetter('string'))
> 
> which is not doing anything to my list unfortunately; it leaves it
> untouched.

Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Timesheet(object):
... def __init__(self, department): self.department=department
... def __repr__(self): return "Timesheet(%s)" % self.department
... 
>>> timesheets = [Timesheet('abc'), Timesheet('def'), Timesheet('acd')]
>>> timesheets
[Timesheet(abc), Timesheet(def), Timesheet(acd)]
>>> import operator
>>> timesheets.sort(key=operator.attrgetter('department'))
>>> timesheets
[Timesheet(abc), Timesheet(acd), Timesheet(def)]

The key bit here being the argument to attrgetter, which is the name
of the attribute to use as they key.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Quoth J Kenneth King :
> mk  writes:
> 
> > Hello everybody,
> >
> > Any better solution than this?
> >
> > def flatten(x):
> > res = []
> > for el in x:
> > if isinstance(el,list):
> > res.extend(flatten(el))
> > else:
> > res.append(el)
> > return res
> >
> > a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
> > print flatten(a)
> >
> >
> > [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
> >
> > Regards,
> > mk
> 
> http://mail.python.org/pipermail/python-list/2005-July/330367.html

That's worth reading.  I'm not sure why I'm finding this fun, but who
cares.  I tried a couple of other functions after reading that article,
and it looks like a generator that scans the nested lists is actually
the winner, and also is in many ways the most elegant implementation.
Of course, as noted in the emails following above article, the test data
is really inadequate for proper optimization testing ;)

-
from __future__ import print_function
from timeit import Timer
from itertools import chain

# This is the one from the article quoted above.
def flatten6(seq):
i = 0
while (i != len(seq)):
while hasattr(seq[i], '__iter__'):
seq[i:i+1] = seq[i]
i = i + 1
return seq


#This is my favorite from a readability standpoint out of
#all the things I tried.  It also performs the best.
def flatten8(seq):
for x in seq:
if not hasattr(x, '__iter__'): yield x
else:
for y in flatten8(x):
yield y



l = [[1, 2, 3], 5, [7, 8], 3, [9, [10, 11, 12], 4, [9, [10, 5, 7]], 1, [5, 
4], 3], 4, 3], 3, 1, 45], 9], 10]]



if __name__=="__main__":
print(l)
print('flatten6', flatten6(l))
print('flatten8', list(flatten8(l)))
print('flatten6', Timer("flatten6(l)", "from temp3 import flatten6, 
l").timeit())
print('flatten8', Timer("list(flatten8(l))", "from temp3 import flatten8, 
l").timeit())


-

>src/python/Python-3.0/python temp3.py
[[1, 2, 3], 5, [7, 8], 3, [9, [10, 11, 12], 4, [9, [10, 5, 7]], 1, [5, 4], 
3], 4, 3], 3, 1, 45], 9], 10]]
flatten6 [1, 2, 3, 5, 7, 8, 3, 9, 10, 11, 12, 4, 9, 10, 5, 7, 1, 5, 4, 3, 4, 3, 
3, 1, 45, 9, 10]
flatten8 [1, 2, 3, 5, 7, 8, 3, 9, 10, 11, 12, 4, 9, 10, 5, 7, 1, 5, 4, 3, 4, 3, 
3, 1, 45, 9, 10]
flatten6 32.8386368752
flatten8 30.7509689331

>python temp3.py 
[[1, 2, 3], 5, [7, 8], 3, [9, [10, 11, 12], 4, [9, [10, 5, 7]], 1, [5, 4], 
3], 4, 3], 3, 1, 45], 9], 10]]
flatten6 [1, 2, 3, 5, 7, 8, 3, 9, 10, 11, 12, 4, 9, 10, 5, 7, 1, 5, 4, 3, 4, 3, 
3, 1, 45, 9, 10]
flatten8 [1, 2, 3, 5, 7, 8, 3, 9, 10, 11, 12, 4, 9, 10, 5, 7, 1, 5, 4, 3, 4, 3, 
3, 1, 45, 9, 10]
flatten6 34.730714798
flatten8 32.3252940178

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Quoth rdmur...@bitdance.com:
> This is all premature optimization, except for the goopy code, which is
> presumably used enough to make it worth optimizing.  And guess what?
> The goopy code wins.  What the people theorizing about the speed of
> extend vs list creation miss is that the things with high overhead in the
> above functions are (1) isinstance and (2) the recursive function call.
> The goopy code avoids this by using type and is, and by unrolling the
> lowest level without a function call.  On the other hand, extend
> _is_ faster than append if you aren't creating a new list, so the
> goopy code can be optimized a little more.
> 
> I remembered the bit about high function call overhead, but the rest
> of it measured:

Oooh, that's embarrassing.  Not only didn't I read the code carefully
enough, I didn't test the actual output of the functions.  The goopy
code doesn't flatten to arbitrary depth, so of course it is faster.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[rdmurray]

Baolong zhen  wrote:
> On Thu, Feb 5, 2009 at 10:17 PM, mk  wrote:
>
> > Brian Allen Vanderburg II wrote:
> > >> def flatten(x):
> > >> res = []
> > >> for el in x:
> > >> if isinstance(el,list):
> > >> res.extend(flatten(el))
> > >> else:
> > >> res.append(el)
> > >> return res
> >
> > >
> > > I think it may be just a 'little' more efficient to do this:
> > >
> > > def flatten(x, res=None):
> > >if res is None:
> > >   res = []
> > >
> > >for el in x:
> > >   if isinstance(el, (tuple, list)):
> > >  flatten(el, res)
> > >   else:
> > >  res.append(el)
> > >
> > >return res
> >
> >
> > Hmm why should it be more efficient? extend operation should not be very
> > costly?
>
> less list creation.

(Took me a while to find the content of your post because
you top posted it.  I've taken the liberty of correcting that.)

This is all premature optimization, except for the goopy code, which is
presumably used enough to make it worth optimizing.  And guess what?
The goopy code wins.  What the people theorizing about the speed of
extend vs list creation miss is that the things with high overhead in the
above functions are (1) isinstance and (2) the recursive function call.
The goopy code avoids this by using type and is, and by unrolling the
lowest level without a function call.  On the other hand, extend
_is_ faster than append if you aren't creating a new list, so the
goopy code can be optimized a little more.

I remembered the bit about high function call overhead, but the rest
of it measured:

temp.py
-
from __future__ import print_function
from timeit import Timer

def flatten1(x):
res = []
for el in x:
if isinstance(el, list):
res.extend(flatten1(el))
else:
res.append(el)
return res

def flatten1a(x):
res = []
for el in x:
if isinstance(el, (tuple, list)):
res.extend(flatten1a(el))
else:
res.append(el)
return res

def flatten1b(x):
res = []
for el in x:
if type(el) is list or type(el) is tuple:
res.extend(flatten1b(el))
else:
res.append(el)
return res


def flatten2(x, res=None):
if res is None:
   res = []
for el in x:
   if isinstance(el, list):
  flatten2(el, res)
   else:
  res.append(el)
return res


def flatten2a(x, res=None):
if res is None:
   res = []
for el in x:
   if isinstance(el, (tuple, list)):
  flatten2a(el, res)
   else:
  res.append(el)
return res


def flatten2b(x, res=None):
if res is None:
   res = []
for el in x:
   if type(el) is list or type(el) is tuple:
  flatten2b(el, res)
   else:
  res.append(el)
return res



def flatten3z(seq):
  lst = []
  for x in seq:
if type(x) is list or type(x) is tuple:
  for val in x:
lst.append(val)
else:
  lst.append(x)
  return lst

def flatten3(seq):
  lst = []
  for el in seq:
if type(el) == list or type(el) is tuple:
  lst.extend(flatten3z(el))
else:
  lst.append(el)
  return lst


def flatten3y(seq):
  lst = []
  for x in seq:
if type(x) is list or type(x) is tuple:
  lst.extend(x)
else:
  lst.append(x)
  return lst

def flatten3a(seq):
  lst = []
  for el in seq:
if type(el) == list or type(el) is tuple:
  lst.extend(flatten3y(el))
else:
  lst.append(el)
  return lst



l = [[1, 2, 3], 5, [7, 8], 3, [9, [10, 11, 12], 4, [9, [10, 5, 7]], 1, [5, 
4], 3], 4, 3], 3, 1, 45], 9], 10]]

cases = dict(
base = Timer(),
c1 = Timer("flatten1(l)", "from temp import flatten1, l"),
c1a = Timer("flatten1a(l)", "from temp import flatten1a, l"),
c1b = Timer("flatten1b(l)", "from temp import flatten1b, l"),
c2 = Timer("flatten2(l)", "from temp import flatten2, l"),
c2a = Timer("flatten2a(l)", "from temp import flatten2a, l"),
c2b = Timer("flatten2b(l)", "from temp import flatten2b, l"),
c3 = Timer("flatten3(l)", "from temp import flatten3, l"),
c3a = Timer("flatten3a(l)", "from temp import flatten3a, l"),
)

if __name__=="__main__":
for (name, case) in sorted(cases.items()):
print("{0:4s} {1}".format(name, case.timeit()))

-

It is also interesting to note that python3.0 is faster in this
particular case (unless there are timing vagrancies on my machine, which
is possible, though the results were fairly consistent over several runs
of the script).  The second run below is using python2.6.1.

>src/python/Python-3.0/python temp.py
base 0.0278329849243
c1   30.4776289463
c1a  44.3886289597
c1b  32.5621030331
c2   25.6131818295
c2a  39.0944678783
c2b  27.1573381424
c3   15.346280098
c3a  14.3178970814

>python temp.py 
base 0.02882695

os.system issues

[rdmurray]

Youri Lammers  writes:
> I want to run a program called 'muscle' with my python script=2C
> muscle uses the following command:
> 'muscle.exe -in filename -out filename'
> so far I got:
> 
> import os
> args = ['-in filename', '-out filename']
> os.system('E:\Programs\muscle\muscle.exe args')
> 
> However when I run this nothing happends no error message nothing.
> 
> So what am I doing wrong here?

You are calling the program muscle with the argument 'args'.

If you want to put your arguments into the system call, you will
want to write it like this:

os.system('E:\Programs\muscle\muscle.exe -in filename -out filename')

If you need to substitute args into the string passed to
os.system, then take a look at python string handling in the
tutorial.

You also will probably want to look at the 'subprocess' module
at some point, which is more flexible than os.system.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 slow file IO

[rdmurray]

Quoth Christian Heimes :
> thomasvang...@gmail.com schrieb:
> > I just recently learned python, I'm using it mainly to process huge
> > <5GB txt files of ASCII information about DNA. I've decided to learn
> > 3.0, but maybe I need to step back to 2.6?
> > 
> > I'm getting exceedingly frustrated by the slow file IO behaviour of
> > python 3.0. I know that a bug-report was submitted here:
> > http://bugs.python.org/issue4533. And a solution was posted.
[snip stuff about patching]
> 
> I suggest you stick with Python 2.5 or 2.6. Python 3.0 isn't as mature
> as the 2.x series. It's brand new, lot's of things have changed, too.

3.1 will come out much sooner than a normal "next dot" release python
would, and will contain very significant improvements in the IO speed.
In the meantime, in your case at least, to get real work done you'll
be better off using 2.6.  You can use the 'from __future__ import' to
bring various 3.0 features into 2.6 and make 2.6 look more like 3.0,
and you can use the -3 flag to check your code for 3.0 compatibility, so
that you will be ready to move back to the 3 series when 3.1 comes out.
(Assuming you don't end up needing any 3rd party libraries that haven't
been ported yet by then, that is.)

Unless, of course, you are comfortable with compiling from source
and would like to help _test_ the 3.1 IO library :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

string replace for back slash

[rdmurray]

"S.Selvam Siva"  wrote:
> I tried to do a string replace as follows,
> 
> >>> s="hi & people"
> >>> s.replace("&","\&")
> 'hi \\& people'
> >>>
> 
> but i was expecting 'hi \& people'.I dont know ,what is something different
> here with escape sequence.

You are running into the difference between the 'repr' of a string (which
is what is printed by default at the python prompt) and the actual
contents of the string.  In the repr the backslash needs to be escaped
by prefixing it with a backslash, just as you would if you wanted to
enter a backslash into a string in your program.  If you print the string,
you'll see there is only one backslash.  Note that you didn't need to
double the backslash in your replacement string only because it wasn't
followed by a character that forms an escape...but the repr of that
string will still have the backslash doubled, and that is really the
way you should write it in your program to begin with for safety's sake.

Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s="hi & people"
>>> replacementstring = "\&"
>>> replacementstring
'\\&'
>>> print replacementstring
\&
>>> x = s.replace("&","\\&")
>>> x
'hi \\& people'
>>> print x
hi \& people

--
http://mail.python.org/mailman/listinfo/python-list

Couple of noobish question

[rdmurray]

Quoth Catherine Heathcote :
> all goes well. I have an idea for a small project, an overly simplistic 
> interactive fiction engine (well more like those old choose your own 
> adventure books, used to love those!) that uses XML for its map files. 
> The main issues I see so far is the XML parsing (I should pick that up 
> ok, I have a blackbelt in google-foo), but more importantly splitting 

Others have answered your other question, but I thought I'd mention
that you will probably want to take a look at the ElementTree module.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Extracting file from zip archive in Python 2.6.1

[rdmurray]

Quoth Brandon Taylor :
> Ok, the first thing I needed to do was add:
> 
> from __future__ import with_statement at the beginning of my file
> 
> but:
> 
> with zip_file.open(zip_name + '/' + thumbnail_image) as source:
> with open(os.path.join(thumbnail_path,
> thumbnail_image), 'wb') as target:
> shutil.copyfileobj(source, target)
> 
> Returns an error on the first line:
> 
> ZipExtFile instance has no attribute '__exit__'
> 
> Googling this error message is turning up nothing, and there's no
> mention of the exception in the docs. Any thoughts?

Yeah, that means the ZipExtFile object hasn't been extended to
handle the context management protocol.  It would be pretty
simple to roll your own for it.  Take a look at the contextlib
module.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Passing environment variable to "subprocess" causes failure

[rdmurray]

Quoth MRAB :
> davidgo...@davidgould.com wrote:
>  > I'm attempting to run subprocess and passing in an environment
>  > variable. When I do this the child process fails with an error. When I
>  > don't pass an environement variable it runs fine.
>  >
>  > BTW Running this code under Windows XP with Python 2.6.1
>  >
>  > Code A:
>  >
>  > p = subprocess.Popen( ['python', '-V'], env={ 'PYTHONPATH': 'C:/
>  > Documents and Settings/David Gould/workspace/DgTools/Common/Trunk/
>  > Source' } )
>  > print p.communicate()[0]
>  > print p.returncode
>  >
>  > Output:
>  >
>  > None
>  > -1072365564
>  >
>  > Code B:
>  >
>  > p = subprocess.Popen( ['python', '-V'] )
>  > print p.communicate()[0]
>  > print p.returncode
>  >
>  > Output:
>  >
>  > Python 2.6.1
>  > 0
>  >
>  > Any idea why passing an environment variable causes it to fail?
>  > Thanks.
>  >
> Are you sure it's not the path within the dict you're passing that's wrong?

Or there's something missing from the environment that python needs,
causing a non-zero return code when it is missing.  (That seems odd,
though.)

You ought to print communiicate()[1] to see if there is an
accompanying error message that would shed light on the problem.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

global variable confusion

[rdmurray]

"Robert D.M. Smith"  wrote:
> I have a question on global variables and how to use them.  I have 2 files;
> a.py & b.py
> 
> # a.py -
> 
> myvar = { 'test' : '123' }
> 
> # ---
> # b.py -
> 
> from a import myvar
> 
> def test():
>  a.myvar = { 'blah' : '456' }
> 
> # -
> 
> If I *'import a*' & type *'a.myvar'* it prints 'test' & '123'.
> 
> Now, if I *'import b'* & type *'b.test()'* ... does this actually change the
> contents of myvar? It seems I have to do *'print b.myvar'* to see the change
> but really I want to just simply do* 'print myvar'*. I want to be able to
> change myvar anywhere. Please help me grasp this concept.

As someone else told you the code above won't work, because you haven't
imported the 'a' that your 'test' function references.  But let's
ignore that issue.

I'm guessing the confusion you are experiencing is arising from the 
concept of name spaces (a very powerful and ubiquitous concept in Python).
A namespace maps an identifier to a object.  A given identifier exists in
exactly one namespace, but many identifiers can refer to the same object.

When you do 'from a import myvar', you are setting the identifier 'myvar' 
_in the current model's global namespace_ to point to the same object that
'myvar' does in module a.  If, in the current module, you then do, say,
'myvar = 1', what you've done is _rebound_ the identifier 'myvar' in the
current module's global namespace to the integer 1 (an integer object).
You haven't affected anything in module 'a'.

If, on the other hand, you do 'import a', what you've done is bound the
global namespace of module 'a' (an object) to the identifier 'a' in the
current module's global namespace.  If you were to do 'a = 1' in the
current module, then you'd rebind the identifier 'a' to the integer 1.
After that a.myvar would be an error.

If, however, you say 'a.myvar = 1', what you've done is rebound the
identifier 'myvar' _in module a's global namespace_ to the integer 1.  
Anything that subsequently references a.myvar is going to get '1' as
the result.

Now, to take your brain around the final bend, think about what happens 
if one module does 'from a import myvar', and module imported later does 
'import a; a.myvar = 1'.  The first module bound its global identifier
'myvar' to the object a.myvar originally pointed to (a little dictionary
in your example above).  The second module rebinds the identifier myvar in 
module a's global namespace to the integer 1.  But that doesn't affect
the first module's global namespace.  There, the identifier 'myvar'
is still bound _to the original dictionary_.

A modification of your example should make what is going on clearer:

---
# a.py
myvar = {'test': '123'}
---
# b.py
from a import myvar
myvar['test'] = '456'
---

Python 2.6.1 (r261:67515, Jan  7 2009, 17:09:13)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from a import myvar
>>> print myvar
{'test': '123'}
>>> import b
>>> print myvar
{'test': '456'}

The difference between the above and doing 'myvar = 1' is that in the
above code the statement 'myvar['test'] = '456' modifies _the object
that myvar points to_, while 'myvar = 1' modifies _what_ myvar points to.

Study that last sentence until you understand it, and a lot of things
that happen in Python will be much clearer to you (such as what
happens when you pass arguments to a function or method!)

If what I have written doesn't make sense, feel free to ask more 
questions.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: is python Object oriented??

[rdmurray]

Quoth David Cournapeau :
> On Wed, Feb 4, 2009 at 2:36 AM,   wrote:
> >
> > Pretty much, unless maybe the code documents what you're not supposed
> > to access:
> 
> But that's my point: that's just not true for many packages I have
> used - some packages do follow the _ convention, some don't. For
> example, to take an example I am somewhat familiar with: distutils
> does not follow this at all. There is no documentation, and it is
> almost impossible to know what's the API from implementation details,
> the interface is leaking everywhere. Now, distutils is an "old"
> package, but my experience at least showed me that this is relatively
> common.
> 
> There are some relatively well known packages which use a different
> mechanism to clearly separate the intended API, without using the _
> convention, or more exactly in supplement of it.

This whole issue gets more complicated, I think, when you consider that
what is "public" versus "private" in an API depends on the consumer.
That is, you might have a group of modules that use a package-internal
interface, while what the "outside world" is supposed to use is a more
restricted API.  The package-internal interface isn't _private_ in the
sense of hiding things from anybody outside the class or its subclasses,
but neither is it public.

I won't be surprised to hear that languages with data hiding have some
provision for this, but to my mind this is a "one, two, many" situation.
That is, if you have one case, things are trivially simple.  If you have
two cases, you can differentiate them.  But once you have three cases,
you almost certainly have _more_ than three cases...in other words a
problem in generalization.

In my code I generally use _ for stuff that is class-private, and document
the "outside" or "stable" API, and everything else is...whatever it needs
to be.  Perhaps the packages you are thinking of do something similar?

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: what IDE is the best to write python?

[rdmurray]

Quoth a...@pythoncraft.com (Aahz):
> In article ,
> Thorsten Kampe   wrote:
> >* Aahz (2 Feb 2009 09:29:43 -0800)
> >> In article ,
> >> Thorsten Kampe   wrote:
> >>>* Aahz (2 Feb 2009 06:30:00 -0800)
>  In article <874ozd3cr3@benfinney.id.au>,
>  Ben Finney   wrote:
> >a...@pythoncraft.com (Aahz) writes:
> >>
> >> Just to register a contrary opinion: I *hate* syntax highlighting
> >
> >On what basis?
>  
>  It makes my eyes bleed
> >>>
> >>>Ever tried sunglasses?
> >> 
> >> Polarized sunglasses don't work too well with LCD monitors
> >
> >Well, the answer to your issue with syntax highlighting is to use a 
> >/decent/ highlighting colour scheme. Don't use that all 
> >purple/red/green/yellow/blue one. That's for psychedelic trips.
> >
> >Use a decent colour scheme for adults and you will see that you will 
> >benefit from syntax highlighting.
> 
> Then I have the problem of copying around the syntax highlighting
> configuration to every computer I use.

Well, _that's_ easy to fix.  I have a little bash script called
'synchome' that uses rsync to update the home directory on any
of the remote machines on which I work.  I've had to install
rsync on one or two of the boxes, but that's a useful thing to
do anyway.

(Granted, I still have a couple bugs to work out, where I haven't
taken the time to conditionalize things properly for some of
the more exotic machine configurations, but hey, if I spend
more time on those machines I'll get around to it...)

Now, if you are a Windows user, then I sympathise...a little  :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python Global State

[rdmurray]

Quoth MRAB :
> er wrote:
> > Simple question, I think: Is there a way to make a completely global
> > variable across a slew of modules?  If not, what is the canonical
> > way to keep a global state?  The purpose of this is to try to prevent
> > circular module imports, which just sort of seems nasty.  Thank you!
> > 
> Simple answer: no.

I'm wondering a little about posting this, but...well, the answer actually
isn't "no".  It's just that it's not something you should _ever_ do.
The trick is to assign the value to the __builtins__ module...

--RDM

PS: I know this because Zope used to do it.  Fortunately they wised
up and refactored that code to be more sane.

--
http://mail.python.org/mailman/listinfo/python-list

Combining several text files

[rdmurray]

Quoth Eric :
> This is my first post, so please advise if I'm not using proper
> etiquette. I've actually searched around a bit and while I think I can
> do this, I can't think of a clean elegant way. I'm pretty new to
> Python, but from what I've learned so far is that there is almost
> always an easier way.
> 
> I have to parse several log files. I've already written a working
> parser. The log files are simple text files that when they get to a
> certain size are renamed to append a number. So, you might end up
> with:
> 
> filename.log.2
> filename.log.1
> filename.log
> 
> The higher the number, the older the file. I want to search for all
> the files in a directory with "filename.log" as part of their name.
> Then I can do one of two things. First I could combine them so that
> the resulting file ends up with the oldest on top and newest on the
> bottom. Otherwise, I could just iterate over the multiple files within
> my parser.
> 
> I don't need working code (that makes things too easy), just clear
> suggestions to a Python newcomer to speed me on my way.

My first thought would be to do something like this (assuming
you are on a unix variant):

yourscript `ls -tr filename.log*`

and then in 'yourscript' do:

import fileinput
for line in fileinput.input():
#process the lines

This has the nice advantage of giving you access to the source filename
and line number within its source file of each line, in case that
is useful.

This is an example of why Python is referred to as "Batteries Included" :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: What is wrong in my list comprehension?

[rdmurray]

Quoth Stephen Hansen :
> I just think at this point ".find" is just not the right method to use;
> "substring" in "string" is the way to determine what he wants is all.
> ".find" is useful for when you want the actual position, not when you just
> want to determine if there's a match at all. The way I'd clean it is to
> remove .find, personally :) I don't remember the outcome of their discussion
> on py-dev, and haven't gotten around to loading up Py3 to test it out :)


Python 3.0 (r30:67503, Dec 18 2008, 19:09:30) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> help(''.find)
Help on built-in function find:

find(...)
S.find(sub[, start[, end]]) -> int

Return the lowest index in S where substring sub is found,
such that sub is contained within s[start:end].  Optional
arguments start and end are interpreted as in slice notation.

Return -1 on failure.


--
http://mail.python.org/mailman/listinfo/python-list

Re: Membership of multiple items to a list

[rdmurray]

My client can handle your Mime and shows me the text part of the signed
message.  It's not as pretty as just seeing an unsigned text message,
but that's a client problem, not yours :)

I would like to think that all newsreader clients could handle mime at
this point, but who knows.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: is python Object oriented??

[rdmurray]

Quoth "Hendrik van Rooyen" :
> Now there are a LOT of dicey statements in the above passionate 
> plea - python is a language, and not a philosophy, but I won't go 
> into that, as that would lead off onto a tangent, of which there have
> been a surfeit in this thread.

Ah, now I understand.  This is where you part company from the
people you are arguing against.

Python _is_ a philosophy.

There is no smiley after that statement.  :)

--RDM

PS: More accurately, Python _embodies_ a philosophy, and to advocate
changes that go against that philosophy is to advocate changing
Python into something that would no longer be Python.  You can do
that, but you aren't likely to get the community to follow you.

--
http://mail.python.org/mailman/listinfo/python-list

Re: is python Object oriented??

[rdmurray]

Quoth "Hendrik van Rooyen" :
>  wrote:
> 
> > 
> > You, sir, should be programming in some language other than Python.
> 
> Why?  - Python is object oriented, but I can write whole systems
> without defining a single class.
> By analogy, if data hiding is added to language, I could write a 
> whole system without hiding a single item.
> Conversely, the lack of data hiding is perceived as a weakness,
> and IMO harms the advocacy of the language - This is proven
> by your statement above, whereby you are driving a user away,
> simply because the language, in one small aspect, does not
> give him what he wants, and the tenor of this thread has been
> very much: "That's how it is - like it or lump it", and no amount
> of careful explanation of why people want the feature has cut 
> any ice - It has all the time been countered with statements
> about how the proponents of private attributes "don't need it",
> (as if they are plain dumb), that an underscore convention is 
> "just as good", (It isn't), that you could crack it if you really 
> tried, (so what?), and other more specious reasons.
> 
> This is IMO an arrogant attitude - 

My apologies!!  There was a smiley missing from the end of that statement
above.  However, you are right, even with the smiley added it would
still be arrogant, and I regret that.

The problem is that there is a fundamental philosophical divide here,
and I don't think the advocates on either side are going to convince
the other.

This debate has happened many times.  

--RDM


--
http://mail.python.org/mailman/listinfo/python-list

Re: Membership of multiple items to a list

[rdmurray]

Quoth Steven D'Aprano :
> On Sun, 01 Feb 2009 12:01:11 -0800, Stephen Hansen wrote:
> 
> >  > style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt
> > 0.8ex; padding-left: 1ex;"> I'd like to know how to elegantly check
> > a list for the membership of any of its items to another list.
> >  Not caring for elegance, I would use the following code:
> > That's one of the useful properties of
> > sets:>>> a = [1,2,3]>>> b =
> > [3,4,5,6]>>> set(a) &
> > set(b)set([3])>>>
> > set(a).intersection(b)set([3]) That's two spellings of the
> > same thing. As for testing: an empty set like an empty list will return
> > false, so "if set(a) & set(b):" will be true or false based on if
> > there's any commonalities between the two
> > lists.--Stephen
> 
> 
> Stephen, do you see the utter mess your posts look like to some others?

I don't even see Stephen Hansen's posts.  My newsreader just shows the
header and says "[HTML part not displayed]".

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with slow httplib connections on Windows (and maybe other platforms)

[rdmurray]

Quoth Christoph Zwerschke :
> rdmur...@bitdance.com schrieb:
> > Quoth Christoph Zwerschke :
> >>With Py 2.3 (without IPv6 support) this is only the IPv4 address,
> >>but with Py 2.4-2.6 the order is (on my Win XP host) the IPv6 address
> >>first, then the IPv4 address. Since the IPv6 address is checked first,
> >>this gives a timeout and causes the slow connect() call. The order by
> >>which getaddrinfo returns IPv4/v6 under Linux seems to vary depending
> >>on the glibc version, so it may be a problem on other platforms, too.
> > 
> > Based on something I read in another thread, this appears to be a problem
> > only under Windows.  Everybody else implemented the TCP/IP stack according
> > to spec, and the IPV6 connect attempt times out immediately, producing
> > no slowdown.
> > 
> > Microsoft, however
> 
> The order in which getaddrinfo returns IPv4 and IPv6 is probably not 
> written in the specs (Posix 1003.1g and RFC 2553). The fact that Windows 
> returns IPv6 addresses first is not wrong in itself.
> 
> For this discussion, see also
> http://www.ops.ietf.org/lists/v6ops/v6ops.2002/msg00869.html
> https://bugzilla.redhat.com/show_bug.cgi?id=190495
> 
> But yes, I also wonder why the connect to the IPv6 loopback address does 
> not time out more quickly on Windows.

Right, it's not the order of the returned items that's the Microsoft
weirdness, it's the long timeout on an attempt to connect to something
that doesn't exist.  There was a long discussion about this, and it might
even have been on python-dev, but I can't lay my hands on the thread.
In short, Microsoft retries and waits a while when the far end says
"no thanks" to a connection attempt, instead of immediately returning
the connection failure the way Linux and etc and etc do.  This applies
to IPV4, too.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Problem with slow httplib connections on Windows (and maybe other platforms)

[rdmurray]

Quoth Christoph Zwerschke :
> What actually happens is the following:
> 
> * BaseHTTPServer binds only to the IPv4 address of localhost, because
>it's based on TCPServer which has address_family=AF_INET by default.
> 
> * HTTPConnection.connect() however tries to connect to all IP addresses
>of localhost, in the order determined socket.getaddrinfo('localhost').
> 
>With Py 2.3 (without IPv6 support) this is only the IPv4 address,
>but with Py 2.4-2.6 the order is (on my Win XP host) the IPv6 address
>first, then the IPv4 address. Since the IPv6 address is checked first,
>this gives a timeout and causes the slow connect() call. The order by
>which getaddrinfo returns IPv4/v6 under Linux seems to vary depending
>on the glibc version, so it may be a problem on other platforms, too.

Based on something I read in another thread, this appears to be a problem
only under Windows.  Everybody else implemented the TCP/IP stack according
to spec, and the IPV6 connect attempt times out immediately, producing
no slowdown.

Microsoft, however

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: is python Object oriented??

[rdmurray]

Quoth thmpsn@gmail.com:
> Anyway, it doesn't matter. We're losing the point here. The point is
> that language support for private access, by disallowing user access
> to private data, provides an unambiguous information hiding mechanism
> which encourages encapsulation. Python's approach, however, which is
> only a naming convention rather than a language feature, merely TRUSTS
> the programmer not to break encapsulation. And sure, if we're all good
> programmers, everything will go well and we'll end up with an
> organized application. But the danger is still there: at any given
> time, we COULD break encapsulation!

You, sir, should be programming in some language other than Python.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleXMLRPCServer question

[rdmurray]

Quoth flagg :
> Let me see if i can elaborate on the requirements.  I have 20+
> different zone files.  I want the xmlrpc server to be able to
> determine what zone file to open by looking at the incoming xml
> request.  For example one of the functions I have now is to show a DNS
> record (I am using dnspython for most of this work)
> 
> If i send an xmlrpc request that uses the 'showRecord' function with
> params of 'oracle1.foo.bar.com'  I want to parse the "params" piece
> and then instruct the xml-rpc server to open foo.bar.com.zone for
> reading.  The reason why i was looking at do_Post() and _dispatch was
> to attempt to read the incoming params and do exactly that.
> 
> Do you think there is an easier way of accomplishing this, than the
> way I am going about it?

Yeah.  Take a look at the SimpleXLMRPC documentation:

http://docs.python.org/library/simplexmlrpcserver.html

In addition to setting up the server as instructed (let's assume you
assign it to the variable 'server'), you just need to do something like:

class MyFunctions:

def showRecord(self, dnsname):
domain = '.'.join(dnsname.split('.')[1:])
with open(domain+'.zone')) as myfile:
# do stuff with data from myfile

server.register_instance(MyFunctions())


You would modify the body of that function to meet your processing
requirements, of course.  That is, SimpleXMLRPCServer does the request
parsing for you, calls the correspondingly named method, and passes it
the params as method arguments.

(This example is taken from the SimpleXMLRPC documentation, I just
selected one particular way of exposing the methods: via a class
instance).

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: glob.fnmatch (was "search speed")

[rdmurray]

Quoth Tim Chase :
> rdmur...@bitdance.com wrote:
> > What you want is:
> > 
> > from fnmatch import fnmatch
> 
> Oh, that's head-smackingly obvious now...thanks!
> 
> My thought process usually goes something like
> 
> """
> I want to do some file-name globbing
> 
> there's a glob module that looks like a good place to start
> 
> hmm, dir(glob) tells me there's a fnmatch thing that looks like 
> what I want according to help(glob.fnmatch)
> 
> oh, the fnmatch() function is inside this glob.fnmatch thing
> 
> so, I want glob.fnmatch.fnmatch()
> """
> 
> It never occurred to me that fnmatch was its own importable 
> module.  

I did a help(glob), saw that fnmatch wasn't in there, did a dir(glob),
saw fnmatch and was puzzled, so I looked up the glob doc page on
docs.python.org for glob.  There was a cross reference at the bottom
of the page to fnmatch, and it was only at that point that I went:
"oh, duh" :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: search speed

[rdmurray]

Quoth Tim Chase :
> PS:  as an aside, how do I import just the fnmatch function?  I 
> tried both of the following and neither worked:
> 
>from glob.fnmatch import fnmatch
>from glob import fnmatch.fnmatch
> 
> I finally resorted to the contortion coded below in favor of
>import glob
>fnmatch = glob.fnmatch.fnmatch

What you want is:

from fnmatch import fnmatch

fnmatch is its own module, it just happens to be in the (non __all__)
namespace of the glob module because glob uses it.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: More mod_wsgi weirdness: process restarts on redirect

[rdmurray]

Quoth Ron Garret :
> In article ,
>  Joshua Kugler  wrote:
> 
> > Ron Garret wrote:
> > > My question is: is this supposed to be happening?  Or is this an
> > > indication that something is wrong, and if so, what?
> > 
> > You are probably just hitting a different instance of Apache, thus the
> > different process ID.
> 
> Yep, that's what it turned out to be.  I thought I had a 
> WSGIDaemonProcess processes=1 directive in my config, but I had it in 
> the wrong place (a different vhost) so it wasn't actually active.
> 
> But that leaves me wondering why the redirect would reliably trigger 
> switching processes.  The reason I thought that I had the correct 
> configuration and only had one process is that when I reloaded the 
> non-redirected page I *always* got the same process ID.  How does 
> mod_wsgi decide which process  (and which thread for that matter) to use?

My WAG would be that when doing a refresh your client used a persistent
http connection, and thus talked again to the same Apache child, but when
it got a redirect it dropped the old connection and opened up a new one,
thus having a high probability of hitting a new child.  But that is,
as I say, a WAG.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

SimpleXMLRPCServer question

[rdmurray]

Quoth flagg :
> I am working on a very basic xmlrpc server, which will expose certain
> functions for administering BIND zone files.  The big problem I am
> having is parsing the incoming xmlrpc request.  Basically part of the
> xmlrpc request will help deterime which zone file is edited.I have
> been looking at the do_POST() method from SimpleXMLRPCDispatcher and
> it seems like I could code something which overides that method,
> however I am new to programming am at a loss, i might even be making
> this tougher than it needs to be.   Is there a way to parse the
> incoming xmlrpc request before the function it is calling is executed?

Wouldn't you be overriding '_dispatch' on SimpleXMLRPCServer instead?

It's been a while since I looked at XMLRPC, but why wouldn't you have
a operation names, with the zone file to operate on as an argument?
Then you'd just be defining your operations as methods on your
SimpleXMLRPCServer, and they'd use the argument to determine which file
to operate on.

If that doesn't make sense, either I've forgotten how XMLRPC works,
or you should explain your requirements a bit more.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Importing modules

[rdmurray]

Quoth Mudcat :
> [attribution omitted by Mudcat]
> > I think you've probably had issues with circular imports (i.e. mutual
> > dependencies), unless you can precisely remember what you were doing and
> > what went wrong.
>
> That's possible, but circular imports become more of a hazard if you
> have to import in several locations. Unify that to one file, and those
> problems are much easier to avoid.

Personally I've never run into circular import problems when importing
library functions.  Then, again, I always use 'from x import y' (or
'y as z' for those modules that use non-unique names for things, like
os.path.split).

Where I've run into circular import problems when when _my_ modules
were circularly importing things.  Which just meant that my code needed
some refactoring.

> > I can make up three or four different logical groupings in my
> > applications... so what is 'logical' could not be the same for everyone,
> > or from every point of view.
>
> That's not the point. The point is that multiple imports can be a
> limiting factor if the module imports don't happen to align with the
> model they'd like to use for their file layout.

I'm inclined to agree with you that organizing code around which other
modules it imports is (usually) not the optimal organization.  But it is
worth _considering_, because sometimes it does reveal a useful pattern
to refactor by.

> However playing around with the files, I guess it is possible to
> create a file that just does imports and then reference them all just
> like you would any other extension of the namespace. I created a file
> called imports and was able to access the sys module within it by
> importing all from imports and calling sys. That way at least all you
> have to do is import the one file each time.

If you are talking about making a file and then doing from X import *,
I'd shudder a little bit looking at such code.  It just doesn't feel
like the python way :).  Which doesn't mean you shouldn't do it, if it
works for you, it is more of a personal preference thing.

What I'd respond to better (and I almost started doing this in my
code...but it's more typing so in the end I didn't) is to have a module,
say 'lib', and just do 'import lib'.  Then everything is called through
'lib': lib.os.path.split('/some/path'), etc, etc.

The difference is that that is explicit.  If I'm reading your code
and I see that, I look at the top and I see the import for 'lib', and I
know to look for a lib module.  If you use the from X import * pattern,
I lose that ability.  Sure, I can look at the top, and see the 'from X
import *', and if that's the only 'from *' import I can guess that the
functions not defined in the file come from that one...but all of that
takes more brain cycles.  It just doesn't feel as clean :)

But, in the end, it is mostly personal preference.  I'm sure there are
Pythonistas out there whose lips will curl when seeing
'from os.path import split as pathsplit, join as pathjoin' in my code :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: verilog like class w/ bitslicing & int/long classtype

[rdmurray]

Quoth Stef Mientki :
> Marc 'BlackJack' Rintsch wrote:
> > On Fri, 30 Jan 2009 00:25:03 +0100, Stef Mientki wrote:
> >
> >   
> >> try this:
> >>
> >> class MyRegClass ( int ) :
> >>   def __init__ ( self, value ) :
> >> self.Value = value
> >>   def __repr__ ( self ) :
> >> line = hex ( self.Value )
> >> line = line [:2] + line [2:].upper()
> >> return line
> >> 
> >
> > def __repr__(self):
> > return '0x%X' % self.value
> >
> >   
> >>   __str__ = __repr__
> >> 
> >
> > This is unnecessary, the fallback of `__str__` is `__repr__` already.
> >
> >   
> well this is what my Python is doing:
> 
> without  _str__ = __repr__
>  >>print shadow_register
> 170
> 
> with  _str__ = __repr__
>  >>print shadow_register
> 0xAA


>>> '__str__' in dir(170)
True

That is, the superclass (int) has an __str__, so you have to override it.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: New Python 3.0 string formatting - really necessary?

[rdmurray]

Quoth Steven D'Aprano :
> Whether using % or format(), I don't see the need to change the code, 
> only the strings.
> 
> Using positional arguments is not really that different:
> 
> "{0} {1}".format("dead", "parrot")
> "{0} {1}".format("perroquet", "mort")

This should be something like:

_("{0} {1}").format(_("dead"), _("parrot"))

where il8n would substitute the template "{1} {0}" when doing French.

> versus:
> 
> "%s %s" % ("dead", "parrot")
> "%s %s" % ("perroquet", "mort")
> 
> In this case, the template on the left remains the same, you just have to 
> reorder the string arguments on the right. Clearly less satisfactory than 
> the solution using keyword substitution, but whatever solution you pick, 
> I don't see any advantage to format() over % formatting. Can you show an 
> example?

Not less satisfactory, but rather unworkable.  You can't do proper il8n
with %s formatting, since there is no way for the il8n machinery to
reorder the argument tuple.  It can only translate the template string.
So when doing il8n, the {} syntax wins out for brevity over the equivalent
% syntax (%s(somename)s).

Not that brevity is that important an argument.  The new system is just
so much more flexible than the old.  As someone else said, the design
is beautiful :)  I have a couple thousand lines of python code I wrote
a while back to layer on a system with this kind of flexibility...I was
shocked and pleased when I saw the PEP, since it echoed so many of the
ideas I had implemented in that code, plus more.  And all done better
of course :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: New Book: Programming in Python 3

[rdmurray]

Quoth Steven D'Aprano :
>The second exception is if the word ends with an S. In British English,
>you put the apostrophe after the S:
>
>Thomas' approach is wholly practical.
>
>In American English, they often (but not always) add an extra S:
>
>Thomas's approach is wholly practical.
>
>which in my opinion is logical but ugly and should be avoided.

I disagree.  For indicating the possessive of the plural ("the books'
spines were damaged), putting the apostrophe after makes perfect sense.
However, when a word like "Thomas" is made possessive in American English,
it is pronounced differently (eg: "Thomases").  In that case, spelling
it Thomas's makes it read more like it sounds.  So you may find it ugly,
but I find it reads better to my ear :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: confused about __str__ vs. __repr__

[rdmurray]

Quoth "Diez B. Roggisch" :
> Neal Becker wrote:
> 
> > Tino Wildenhain wrote:
> > 
> >> Neal Becker wrote:
> >> ...
> >>> That makes no sense to me.  If I call 'print' on a container, why
> >>> wouldn't it recursively  print on the contained objects?  Since print
> >>> means call str, printing a container should recursively call str on the
> >>> objects.
[...]
> >> Every class is free on how to best implement __str__, you will find
> >> the same behavior on tuple and list as well.
> >> 
> >> Maybe its discussable to change the implementation sensibly, best if you
> >> would come with a proposal? Perhaps pprint.pprint is a starting point?
> >> 
> >> Regards
> >> Tino
> > 
> > First, I'd like to know if there is a rationale for the current design. 
> > Am I correct in thinking this is a defect?
> 
> I don't think so. First of all, there is no "generic" way of printing a
> collection. And the current implementation tries to give an overview what
> is contained in the collection, without trying to make it "fancy" - any
> such thing needed to be hand-coded anyway.
> 
> Using repr for that is better suited, as for example string keys are printed
> with quotes around them - making clear what they are, and not irritating
> the user through potentially contained spaces or even things that look as
> if they are python objects.
> 
> For example, if repr *wasn't* used,
> 
> { "{foo=bar}" : "baz"}
> 
> would be printed
> 
> {{foo=bar} : baz}
> 
> Which is *not* what the dictionary actually contains!
> 
> The same goes for unicode-objects. They appear with their "funny" characters
> as \xXX-codes - instead of bailing out on you with unicode-errors.

Note that this last is no longer true in Python 3.0.  There, repr prints
into the current encoding of the output, and so you may get decode errors.
This is an accepted usability tradeoff, as it allows people using these
character sets (and who therefore will have terminals set to display
them!) to actually see the characters in their data.  If you need the
old behavior you use 'ascii'.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: getting object instead of string from dir()

[rdmurray]

Quoth Rominsky :
> vars seems to give an identical response as locals and globals, at
> least in my test name space.  All three are new commands for me.  I

Without arguments vars() returns the same thing as locals().

> like the idea of adopting either vars or locals instead of dir as it
> sets up the value for me to use.  I will play with them both a little
> more and read up on them to learn there uses and limitations.  The key
> step for me is still to be able to automatically query the key names
> from vars or locals for what type the variable is.  In my previous
> post I discussed using eval and the string in the key name such as
> 
> eval('type(%s)'%vars().keys()[0])

>>> eval('type(%s)'%vars().keys()[0])

>>> vars().keys()[0]
'__builtins__'
>>> vars().values()[0]

>>> type(vars().values()[0])

>>> for name, obj in vars().items():
... print "%s (%s)" % (name, type(obj))
... 
__builtins__ ()
__name__ ()
__doc__ ()
__package__ ()

--
http://mail.python.org/mailman/listinfo/python-list

Why no lexical scoping for a method within a class?

[rdmurray]

Quoth walterbyrd :
> For a language as well structured as Python, this seems somewhat
> sloppy, and inconsistant.  Or is there some good reason for this?

Yes.  It's called Object Oriented Programming.

> Here is what I mean:
> 
> def a():
> x = 99
> print x
> 
> def b():
> print x
> 
> a()
> b() # raises an exception because x is not defined.
> 
> However in the methods are within a class, the scoping seems to work
> differently.
> 
> class ab():
> def a(self):
> self.x = 99
> print self.x
> def b(self):
> print self.x
> 
> i = ab()
> i.a()
> i.b() # this works, why no lexical scoping?

Because x is an attribute.

If you don't understand what that means, read any introductory
article on OOP.

To give you a clue, if you had said:

class ab():
def a(self):
x = 99
print x
def b(self):
print x

You'd have gotten the exception you expected (assuming x wasn't
defined globally).

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: help I'm getting delimited

[rdmurray]

Quoth John Machin :
> On Dec 18, 1:28 am, aka  wrote:
> >  @expose(allow_json=True)
> 
> Means what? Does what?
> Does the problem still happen without that?

Means what he's posting is not a standalone script :)

He says it's part of his turbogears ap.  @expose says that this
method is callable by name from a URL, and allow_json means
it can be called with a parameter requesting a json formatted
response instead of html.

> Funny, the indentation changed there --- for the very last time, is
> that the actual code of a standalone script that reproduces the
> problem?

Alex, I would strongly suggest that you move your code out into a
standalone script and debug it there (you'll get more help from this
group if you do, for one thing!).  After you get it working
standalone you can incorporate it back into your Turbogears ap.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

eval() and global variables

[rdmurray]

Quoth "=?ISO-8859-1?Q?Juan_Pablo_Romero_M=E9ndez?=" :
> Hello,
> 
> Suppose this function is given:
> 
> def f(x,y):
>   return x+y+k
> 
> Is it possible to somehow assign a value to k without resorting to
> making k global?
> 
> I'm thinking something like this:
> 
> eval("f(1,1)", {"f":f, "k":1})
> 
> Or even better, something like:
> 
> def g(k):
>   return f
> 
> g(1)(1,1) ==> 3

>>> def g(k):
... def f(x,y):
... return x+y+k
... return f
... 
>>> g(1)(1,1)
3

But what's the use case?  The above might not satisfy it.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Generator slower than iterator?

[rdmurray]

Quoth Lie Ryan :
> On Tue, 16 Dec 2008 12:07:14 -0300, Federico Moreira wrote:
> 
> > Hi all,
> > 
> > Im parsing a 4.1GB apache log to have stats about how many times an ip
> > request something from the server.
> > 
> > The first design of the algorithm was
> > 
> > for line in fileinput.input(sys.argv[1:]):
> > ip = line.split()[0]
> > if match_counter.has_key(ip):
> > match_counter[ip] += 1
> > else:
> > match_counter[ip] = 1
> 
> nitpick:
> dict.has_key is usually replaced with 
> if ip in match_counter: ...

I'd say it's just slightly more than a nitpick nowadays:  since Python
3.0 does not support has_key, it's time to get used to not using it :)

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Having Issues with CMD and the 'python' command

[rdmurray]

On Mon, 15 Dec 2008 at 23:01, James Mills wrote:

On Mon, Dec 15, 2008 at 10:51 PM, Lamonte Harris  wrote:

Every time I start cmd on windows it requires me to "set
path=%path%;C:\python26" why? I'm getting annoyed...


"cmd" has _nothing_ to do with Python.



(Top posting corrected.)

But the answer is that you need to update your PATH string at the system
level.  You do that in Control Panel/System/Advanced/Environment variables
(it's a button on the advanced screen, which is something that confused
me the first time I went looking for it).

ObPython: you know, it occurs to me that Windows follows exactly the
opposite philosophy from Python when it comes to hierarchy.  Python's
Zen is "shallow is better than deep", whereas Windows' philosophy
is "deep is better than shallow".  Every release of Windows seems
to bury the things one needs to do to administer the system deeper
and deeper inside a nested set of windows...and every time I touch
Windows I am reminded how sensible the Python Zen is :)

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: %s place holder does not let me insert ' in an sql query with python.

[rdmurray]

On Mon, 15 Dec 2008 at 18:16, Krishnakant wrote:

how do you let the ' go as a part of the string?
I have used %s as placeholder as in
queryString = "insert into venders values ('%s,%s,%s" %
(field1,field2,field3 ) ...
This is not working for the ' values.


This is untested, but I think what you want is:

cursor.execute("insert into venders values (?, ?, ?)", field1, field2,
field3)

This uses parameter binding and should properly quote the values.
It's also the "right way" to do it to avoid sql injection attacks
and for efficiency if you run the same query multiple times.

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess to pipe through several processes?

[rdmurray]

On Sat, 13 Dec 2008 at 23:05, Neal Becker wrote:

How would I use suprocess to do the equivalent of:

cat - | program_a | program_b


Have you tried extending the pipe example from the manual?

http://docs.python.org/library/subprocess.html#replacing-shell-pipeline

--David
--
http://mail.python.org/mailman/listinfo/python-list

Re: Shorter tracebacks

[rdmurray]

On Sat, 13 Dec 2008 at 06:13, bearophileh...@lycos.com wrote:

When I write recursive code in Python I sometimes go past the maximum
allowed stack depth, so I receive a really long traceback. The show of
such traceback on my screen is very slow (despite a CPU able to
perform billions of operations each second). So I think I'd like
something to shorten them.
I am thinking about something like:
from __future__ import short_traceback

That allows me to see only the first and last parts of the stack
trace, and skips the (generally very redundant) middle part.


_If_ such a feature were to be added, it should be a runtime option
to the interpreter, not a __future__ import.

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Interface & Implementation

[rdmurray]

On Fri, 12 Dec 2008 at 16:07, J Ramesh Kumar wrote:

I am new to python. I require some help on implementing interface and
its implementation. I could not find any sample code in the web. Can
you please send me some sample code which is similar to the below java
code ? Thanks in advance for your help.


public interface MyIfc
{
   public void myMeth1();
   public void myMeth2();
}

public class MyClass implements MyIfc
{
 public void myMeth1()
 {
 //do some implementation
 }

public void myMeth2()
  {
  //do some implementation
  }
}
...

MyClass myc=new MyClass();
Hashtable hash=new Hashtable();
hash.put("MYIFC",myc);

MyIfc myi=(MyIfc)hash.get("MYIFC");
myi.myMeth1();
myi.myMeth2();
.


The python 2.x way to to this would be:

class MyClass(object):

def myMeth1(self):
#do some implementation

def myMeth2(self):
#do some implementation

myc = MyClass()
hash = dict()
hash["MYIFC"] = myc
myi = hash["MYIFC"]
myi.myMeth1()
myi.myMeth2()

Which is to say, python 2.x does not have any formal notion
of interfaces.

In python 3.0 if you have a need for an interface you can do this:

from abc import ABCMeta, abstractmethod

class MyIfc:
__metaclass__ = ABCMeta

@abstractmethod
def myMeth1(self): pass

@abstractmethod
def myMeth2(self): pass


class MyClass(MyIfc):
[from here on just like above]

Note however, that an Abstract Base Class is _not_ the same thing
as a Java interface, though it can serve some of the same purposes.
See http://docs.python.org/dev/3.0/whatsnew/2.6.html#pep-3119 for
more.

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Preventing execution of a method

[rdmurray]

On Thu, 11 Dec 2008 at 13:41, Emanuele D'Arrigo wrote:

On Dec 11, 7:48?pm, Bruno Desthuilliers
 wrote:

or to provide read-only
access. I.e. right now I'm working on the graphical client which
potentially could be rewritten entirely by the users. It is necessary
and perfectly reasonable for the client module to access some of the
objects to be represented graphically, but those objects shouldn't be
modifiable by it.


Why so ? At worst, they'll break everything.


-IF- the application was single-user yes, it wouldn't be a big deal.
But as it is potentially multi-user, I don't want one party to corrupt
the application for everybody else. Say I'm writing a multi-player
version of a card game (I'm not). For the sake of the argument let's
imagine that the players are all playing on the same computer, taking
turns. However, they are using a GUI written by one of the players who
unbeknown to them, has written it to modify the game state
appropriately and give himself the right cards at the right time. If
the game state is read-only and can only be queried but not modified
by the GUI, the player can rewrite the GUI but cannot cheat. Isn't
this a legitimate concern?


In that case, you've got much bigger problems than a lack of private
methods.  As someone else pointed out, _no_ language is secure in the
face of this requirement.

Proving this kind of interface requires a lot of careful security oriented
thinking.  Whole operating systems have been designed to address these
kinds of issues...

--RDM--
http://mail.python.org/mailman/listinfo/python-list

Re: internal circular class references

[rdmurray]

On Thu, 11 Dec 2008 at 09:33, Ethan Furman wrote:

Carl Banks wrote:

 On Dec 10, 5:26 pm, Ethan Furman  wrote:
 First of all, do you even need to wrap the datetime.date class?  With
 Python's duck typing ability, you could have a separate NullDate class
 to go alongside the datetime.date, and use a regular datetime.date
 object when the date is present, and NullDate when it's absent.  If
 necessary you can subclass datetime.date to add any new methods it
 would have to have.  Use a factory function to return either NullDate
 or a datetime.date depending on whether the dbf cell is empty.

 class ValidDate(datetime.date):
 def is_valid(self):
 return True

 class NullDate(object):
 # implement any necessary methods of datetime.date interface here
 def is_valid(self):
 return False

 def create_date_from_dbf_cell(dbf_cell):
 if dbf_cell.empty():
 return NullDate()
 return ValidDate(dbf_cell.value)


 If you do this, you don't have to muck around with __getattr__ or
 __new__ or snooping to datetime.date's class dict anything like that.


 Carl Banks


Good question.  My goal with NullDate is to have a date object that I can 
treat the same regardless of whether or not it actually holds a date. 
NullDates with no value should sort before any NullDates with a value, should 
be comparable to dates as well as NullDates, and should support all the same 
methods.  In other words, I don't want to have to worry about whether my date 
object has an actual date under most circumstances (printing, using as 
dictionary keys, comparing, etc.).


Does my design make more sense given these expanded requirements, or could it 
still be done simpler?  For that matter, do my requirements make sense?


What Carl is saying is that since python uses duck typing ("if it quacks
like a duck, it is a duck"), all you need to do is make your NullDate
object quack enough like a date to handle your use cases, and then you
can freely mix dates and NullDates _without having to care which one a
given object is_ (python won't care).  (Until you do care, at which
point you can use isinstance to find out if it is a NullDate).

As far as I can see, your requirements as you've outlined them can be met
by mixing date objects and appropriately implemented NullDate objects.
And that way NullDates will _only_ represent null dates, thus making
its name more meaningful :).

To do this you just need to implement on NullDate those methods that
are going to give NullDate the behavior you need: the rich comparison
operators, __str__, __hash__, etc.  Or it might be easier to subclass
date and override some of the methods.

Unless I'm misunderstanding your requirements, of course :)

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question: if var1 == var2:

[rdmurray]

On Thu, 11 Dec 2008 at 10:24, Kirk Strauser wrote:

At 2008-11-29T04:02:11Z, Mel <[EMAIL PROTECTED]> writes:


You could try

for item in fname:
item = item.strip()


This is one case where I really miss Perl's "chomp" function.  It removes a
trailing newline and nothing else, so you don't have to worry about losing
leading or trailing spaces if those are important to you.


>>> '  ab c  \r\n'.rstrip('\r\n')
'  ab c  '
>>> '  ab c  \n'.rstrip('\r\n')
'  ab c  '
>>> '  ab c  '.rstrip('\r\n')
'  ab c  '

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Preventing execution of a method

[rdmurray]

On Thu, 11 Dec 2008 at 08:16, alex23 wrote:

On Dec 12, 2:07?am, "Emanuele D'Arrigo" <[EMAIL PROTECTED]> wrote:

I.e. if I have a class with two methods, doSomethingSafe() and
doSomethingDangerous(), is there a way to prevent another module from
executing doSomethingDangerous() but allow the execution of
doSomethingSafe()?

My understanding is that in python this is not possible. Can you
confirm?


Your understanding is correct.

The Python convention is to prefix non-public methods/classes etc with
an underscore, as in _doSomethingDangerous(). This is meant to
indicate to anyone using your module that they shouldn't use this
function, at least not without having a good understanding of what it
does.


There is, however, also the possibility of prefixing the method name
with '__'.  The invokes 'name mangling', which makes it more difficult
(though not impossible, the idea is to avoid accidents) for the method
to be called from outside the class.

http://www.python.org/doc/2.5.2/tut/node11.html#SECTION001160.

--RDM--
http://mail.python.org/mailman/listinfo/python-list

Re: forcing future re-import from with an imported module

[rdmurray]

On Wed, 10 Dec 2008 at 14:53, _wolf wrote:

thanks for your answer. i am aware that imports are not designed to
have side-effects, but this is exactly what i want: to trigger an
action with `import foo`. you get foo, and doing this can have a side-
effect for the module, in roughly the way that a `from __future__
import with_statement` changes the interpretation of the current
module (of course, i do not intend to effect syntactic changes---my
idea is to look into the module namespace and modify it). think of it
as ?metamodule programming? (? la metaclass programming).

maybe import hooks are the way to go? somtimes it would be good if
there was a signalling system that broadcasts all kinds of system
state change.

cheers & ~flow

ok so the question is: how to make it so each import of a given module
has a side-effect, even repeated imports?


Why can't you have the code that is doing the import subsequently
call a function from the module to produce whatever side effect it
is you want?  Explicit is better than implicit.  A python
programmer is going to expect that importing a module is idempotent,
and breaking that assumption strikes me as a bad idea.

Maybe you should give us more details about what problem it is
you are trying to solve.  There might be a good pythonic way to
solve it.

--RDM--
http://mail.python.org/mailman/listinfo/python-list

Re: SequenceMatcher bug ?

[rdmurray]

On Tue, 9 Dec 2008 at 22:15, eliben wrote:

On Dec 10, 4:12?am, [EMAIL PROTECTED] wrote:

On Mon, 8 Dec 2008 at 23:46, eliben wrote:

This is about Python 2.5.2 - I don't know if there were fixes to this
module in 2.6/3.0



I think I ran into a bug with difflib.SequenceMatcherclass.
Specifically, its ratio() method. The following:



SequenceMatcher(None, [4] + [10] * 500 + [5], [10] * 500 + [5]).ratio
()



returns 0.0



While the same with 500 replaced by 100 returns .99... something
Looking at the code ofSequenceMatcherthere's some caching going on
when the sequences are longer than 200 elements (and indeed, I can
reproduce the bug above 200 but not below). Can anyone confirm that
this misbehaves and suggest a workaround ?


Python 2.5.2 (r252:60911, Sep 29 2008, 20:34:04)
[GCC 4.3.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.>>> from 
difflib importSequenceMatcher

SequenceMatcher(None, [4] + [10] * 500 + [5], [10] * 500 +
[5]).ratio()


0.99900299102691925



Strange. I could reproduce the problem both on ActiveState Python
2.5.2 for Windows, and in the online Try Python evaluator:

http://try-python.mired.org/


My system is Gentoo, which installs python from source.  Maybe gentoo
applies patches that the binary releases don't have.

--RDM--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I manually uninstall setuptools (installed by egg)?

[rdmurray]

On Tue, 9 Dec 2008 at 18:49, [EMAIL PROTECTED] wrote:

On Ubuntu, I accidentally manually installed setuptools
http://pypi.python.org/pypi/setuptools/0.6c9 (by running the .egg file
as a shell script via sudo), and now realize I should just be using
apt to take care of my system Python packages. I also installed one or
two packages using its ``easy_install``.

Looks like it lives in ``/usr/lib/python2.5/site-packages``.

How can I manually remove those packages installed using that
``easy_install``, and then manually remove the setuptools package I
installed?


rm -r /usr/lib/python2.5/site-packages/

Then find the .pth file (in the site-packages directory) that
references the egg, and delete the line referencing the egg.

Setuptools has no uninstall function, as far as I know.

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Text parsing via regex

[rdmurray]

On Mon, 8 Dec 2008 at 16:51, Robert Kern wrote:

Robocop wrote:

 Wow!  Thanks for all the input, it looks like that textwrapper will
 work great for my needs.  And thanks for the regex help everyone.
 Also, i was thinking of using a list, but i haven't used them much in
 python.  Is there anything in python that is equivalent to pushback in
 c++ for vectors?


list.append()


Fun with lists:

>>> l = list()
>>> l.append('a')
>>> l
['a']
>>> l.extend(['b', 'c'])
>>> l
['a', 'b', 'c']
>>> l[1:] = [1, 2, 3]
>>> l
['a', 1, 2, 3]
>>> l.pop()
3
>>> l
['a', 1, 2]
>>> l[:0] = ['z']
>>> l
['z', 'a', 1, 2]

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: SequenceMatcher bug ?

[rdmurray]

On Mon, 8 Dec 2008 at 23:46, eliben wrote:

This is about Python 2.5.2 - I don't know if there were fixes to this
module in 2.6/3.0

I think I ran into a bug with difflib.SequenceMatcher class.
Specifically, its ratio() method. The following:

SequenceMatcher(None, [4] + [10] * 500 + [5], [10] * 500 + [5]).ratio
()

returns 0.0

While the same with 500 replaced by 100 returns .99... something
Looking at the code of SequenceMatcher there's some caching going on
when the sequences are longer than 200 elements (and indeed, I can
reproduce the bug above 200 but not below). Can anyone confirm that
this misbehaves and suggest a workaround ?


Python 2.5.2 (r252:60911, Sep 29 2008, 20:34:04) 
[GCC 4.3.1] on linux2

Type "help", "copyright", "credits" or "license" for more information.

from difflib import SequenceMatcher
SequenceMatcher(None, [4] + [10] * 500 + [5], [10] * 500 +
[5]).ratio()

0.99900299102691925

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to report progress at fixed intervals

[rdmurray]

On Tue, 9 Dec 2008 at 13:27, Slaunger wrote:

On 9 Dec., 19:35, [EMAIL PROTECTED] wrote:


I felt like a little lunchtime challenge, so I wrote something that
I think matches your spec, based on your sample code. ?This is not
necessarily the best implementation, but I think it is simpler and
clearer than yours. ?The biggest change is that the work is being
done in the subthread, while the main thread does the monitoring.


Well, thank you for spending your lunch time break on my little
problem.


It would be fairly simple to enhance this so that you could pass
arbitrary arguments in to the worker function, in addition to
or instead of the loop counter.


Yes, I agree


---
"""
Test module for testing generic ways of displaying progress
information at regular intervals.
"""
import random
import threading
import time

def work(i):
? ? ?"""
? ? ?Dummy process function, which takes a random time in the interval
? ? ?0.0-0.5 secs to execute
? ? ?"""
? ? ?print "Work step %d" % i
? ? ?time.sleep(0.5 * random.random())

class Monitor(object):
? ? ?"""
? ? ?This class creates an object that will execute a worker function
? ? ?in a loop and at regular intervals emit a progress report on
? ? ?how many times the function has been called.
? ? ?"""

? ? ?def dowork(self):
? ? ? ? ?"""
? ? ? ? ?Call the worker function in a loop, keeping track of how
? ? ? ? ?many times it was called in self.no
? ? ? ? ?"""
? ? ? ? ?for self.no in xrange(self.max_iter):
? ? ? ? ? ? ?self.func(self.no)

? ? ?def __call__(self, func, verbose=True, max_iter=20, progress_interval=1.0):

I had to look up the meaning of __call__, to grasp this, but I get
your methology

? ? ? ? ?"""
? ? ? ? ?Repeatedly call 'func', passing it the loop count, for max_iter
? ? ? ? ?iterations, and every progress_interval seconds report how
? ? ? ? ?many times the function has been called.
? ? ? ? ?"""
? ? ? ? ?# Not all of these need to be instance variables, but they might
? ? ? ? ?# as well be in case we want to reference them in an enhanced
? ? ? ? ?# dowork function.
? ? ? ? ?self.func = func
? ? ? ? ?self.verbose = verbose
? ? ? ? ?self.max_iter=max_iter
? ? ? ? ?self.progress_interval=progress_interval

? ? ? ? ?if self.verbose:
? ? ? ? ? ? ?print ("Work through all %d steps reporting progress every "
? ? ? ? ? ? ? ? ?"%3.1f secs...") % (self.max_iter, self.progress_interval)

? ? ? ? ?# Create a thread to run the loop, and start it going.
? ? ? ? ?worker = threading.Thread(target=self.dowork)
? ? ? ? ?worker.start()

? ? ? ? ?# Monitoring loop.
? ? ? ? ?loops = 0
? ? ? ? ?# We're going to loop ten times per second using an integer count,
? ? ? ? ?# so multiply the seconds parameter by 10 to give it the same
? ? ? ? ?# magnitude.
? ? ? ? ?intint = int(self.progress_interval*10)

Is this not an unnecessary complication?

? ? ? ? ?# isAlive will be false after dowork returns
? ? ? ? ?while worker.isAlive():
? ? ? ? ? ? ?loops += 1
? ? ? ? ? ? ?# Wait 0.1 seconds between checks so that we aren't chewing
? ? ? ? ? ? ?# CPU in a spin loop.
? ? ? ? ? ? ?time.sleep(0.1)

Why not just call this with progress_interval directly?


Because then the program make take up to progress_interval seconds to
complete even after all the work is done.  For a long running program
and a short progress_interval that might not matter, so yes, that would
be a reasonable simplification depending on your requirements.


? ? ? ? ? ? ?# when the modulus (second element of divmod tuple) is zero,
? ? ? ? ? ? ?# then we have hit a new progress_interval, so emit the report.

And then avoid this if expression?

? ? ? ? ? ? ?if not divmod(loops, intint)[1]:
? ? ? ? ? ? ? ? ?print "Processed %d of %d" % (self.no, self.max_iter)

? ? ? ? ?if verbose:
? ? ? ? ? ? ?print "Finished working through %d steps" % max_iter

if __name__ == "__main__":
? ? ?#Create the monitor.
? ? ?monitor = Monitor()
? ? ?#Run the work function under monitoring.
? ? ?monitor(work)

I was unfamiliar with this notation, but now I understand it simply
invokes __call__. Thank you for showing me that!


Yes, it is a very nice feature of python :)


OK. I agree this is a more elegant implementation, although I my mind,
I find it more natural if the reporting goes on in a subthread, but


You could pretty easily rewrite it to put the reporter in the subthread,
it was just more natural to _me_ to put the worker in the subthread,
so that's how I coded it.  Note, however, that if you were to write a
GUI front end it might be important to put the worker in the background
because on some OSes it is hard to update GUI windows from anything
other than the main thread.  (I ran into this in a Windows GUI ap I
wrote using wxPython).


that is a matter of taste, I guess. Anyway: Thank you again for
spending your lunch break on this!


No problem, it was fun.

--RDM--
http://mail.python.org/mailman/listinfo/python-list

Re: When (and why) to use del?

[rdmurray]

On Tue, 9 Dec 2008 at 18:55, Duncan Booth wrote:

Albert Hopkins <[EMAIL PROTECTED]> wrote:


def otherfunction():
try:
# some stuff
except SomeException, e:
# more stuff
del e
return


I think this looks ugly, but also does it not hurt performance by
preempting the gc?  My feeling is that this is a misuse of 'del'. Am I
wrong?  Is there any advantage of doing the above?


It is probably a complete waste of time, but there are situations where
code similar to this can be useful:

   def otherfunction():
   try:
   # some stuff
   except SomeException, e:
   # more stuff
   del e
   raise
   return

The point of code like this is that when a function exits by throwing an
exception the traceback includes a reference to the stack frame and all the
local variables. In some situations that can result in large data
structures not being freed for a very long time (e.g. until another
exception is thrown that is handled at the same level). So, *in very
specialised situations* it may be important to delete particular names from
the local namespace.


If I'm reading http://www.python.org/dev/peps/pep-3110/ right, Python
3.0 eliminates even this use case :)

I have had occasions to use del, when I don't want the variable in the
namespace anymore for some reason (for example, when I'm later going to
be scanning the namespace for some reason).

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can't figure out where SyntaxError: can not delete variable 'x' referenced in nested scope us coming from in python >=2.6

[rdmurray]

On Tue, 9 Dec 2008 at 13:11, Albert Hopkins wrote:

Say I have module foo.py:

   def a(x):
  def b():
  x
  del x

[...]

The difference is under Python 2.4 I get a traceback with the lineno and
offending line, but I do not get a traceback in Pythons 2.6 and 3.0.


2.5 also omits the traceback.  Smells like a bug to me.

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to report progress at fixed intervals

[rdmurray]

On Tue, 9 Dec 2008 at 08:40, Slaunger wrote:

I am a novice Python 2.5 programmer, who write some cmd line scripts
for processing large amounts of data.

I would like to have possibility to regularly print out the progress
made during the processing, say every 1 seconds, and i am wondering
what a proper generic way to do this is.

I have created this test example to show the general problem. Running
the script gives me the output:

Work through all 20 steps reporting progress every 1.0 secs...
Work step 0
Work step 1
Work step 2
Work step 3
Work step 4
Processed 4 of 20
Work step 5

[...]

Work step 19
Finished working through 20 steps

[...]

Quite frankly, I do not like what I have made! It is a mess,
responsibilities are mixed, and it seems overly complicated. But I
can't figure out how to do this right.

I would therefore like some feedback on this proposed generic "report
progress at regular intervals" approach presented here. What could I
do better?


I felt like a little lunchtime challenge, so I wrote something that
I think matches your spec, based on your sample code.  This is not
necessarily the best implementation, but I think it is simpler and
clearer than yours.  The biggest change is that the work is being
done in the subthread, while the main thread does the monitoring.

It would be fairly simple to enhance this so that you could pass
arbitrary arguments in to the worker function, in addition to
or instead of the loop counter.

---
"""
Test module for testing generic ways of displaying progress
information at regular intervals.
"""
import random
import threading
import time

def work(i):
"""
Dummy process function, which takes a random time in the interval
0.0-0.5 secs to execute
"""
print "Work step %d" % i
time.sleep(0.5 * random.random())


class Monitor(object):
"""
This class creates an object that will execute a worker function
in a loop and at regular intervals emit a progress report on
how many times the function has been called.
"""

def dowork(self):
"""
Call the worker function in a loop, keeping track of how
many times it was called in self.no
"""
for self.no in xrange(self.max_iter):
self.func(self.no)

def __call__(self, func, verbose=True, max_iter=20, progress_interval=1.0):
"""
Repeatedly call 'func', passing it the loop count, for max_iter
iterations, and every progress_interval seconds report how
many times the function has been called.
"""
# Not all of these need to be instance variables, but they might
# as well be in case we want to reference them in an enhanced
# dowork function.
self.func = func
self.verbose = verbose
self.max_iter=max_iter
self.progress_interval=progress_interval

if self.verbose:
print ("Work through all %d steps reporting progress every "
"%3.1f secs...") % (self.max_iter, self.progress_interval)

# Create a thread to run the loop, and start it going.
worker = threading.Thread(target=self.dowork)
worker.start()

# Monitoring loop.
loops = 0
# We're going to loop ten times per second using an integer count,
# so multiply the seconds parameter by 10 to give it the same
# magnitude.
intint = int(self.progress_interval*10)
# isAlive will be false after dowork returns
while worker.isAlive():
loops += 1
# Wait 0.1 seconds between checks so that we aren't chewing
# CPU in a spin loop.
time.sleep(0.1)
# when the modulus (second element of divmod tuple) is zero,
# then we have hit a new progress_interval, so emit the report.
if not divmod(loops, intint)[1]:
print "Processed %d of %d" % (self.no, self.max_iter)

if verbose:
print "Finished working through %d steps" % max_iter

if __name__ == "__main__":
#Create the monitor.
monitor = Monitor()
#Run the work function under monitoring.
monitor(work)
--
http://mail.python.org/mailman/listinfo/python-list

Re: pretty strange behavior of "strip"

[rdmurray]

On Fri, 5 Dec 2008 at 07:54, Mark Tolonen wrote:

> >  import re
> >  re.split('[,.]','blah,blah.blah')

['blah', 'blah', 'blah']


Thank you.  Somehow it never occurred to me that I could use that
kind of pattern that way.  I guess my brain just doesn't think
in regexes very well :)

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: pretty strange behavior of "strip"

[rdmurray]

On Thu, 4 Dec 2008 at 20:54, Terry Reedy wrote:

 'toc.html'
> > >  test[4].strip('.html')
 'oc'

 Can't figure out what is going on, really.


What I can't figure out is why, when people cannot figure out what is going 
on with a function (or methods in this case), they do not look it up the doc. 
(If you are an exception and did, what confused you?)  Can you enlighten me?


I'm a little embarrassed to admit this, since I've been using python for
many years, but until I read these posts I did not understand how strip
used its string argument, and I _have_ read the docs.  I can't tell you
what confused the OP, but I can tell you what confused me.

I have often wished that in 'split' I could specify a _set_ of characters
on which the string would be split, in the same way the default list
of whitespace characters causes a split where any one (or more) of
them appears.  But instead the string argument is a multi-character
separator.  (Which is sometimes useful and I wouldn't want to lose the
ability to specify a multi-character separator!)

My first experience in using the string argument was with split, so when I
ended up using it with strip, by analogy I assumed that the string passed
to strip would also be a multi-character string, and thus stripped only
if the whole string appeared exactly.  Reading the documentation did
not trigger me reconsider that assumption.  I guess I'm just lucky that
I haven't run into any bugs (but I think I've used the string argument
to strip only once or twice in my career).

It would be lovely if both the split and strip methods would have a
second string argument that would use the string in the opposite sense
(as a set for split, as a sequence match for strip).

In the meantime the docs could be clarified by replacing:

the characters in the string will be stripped

with

all occurrences of any of the characters in the string will be
stripped

--RDM

PS: the OP might want to look at th os.path.splitext function.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why shouldn't you put config options in py files

[rdmurray]

On Thu, 4 Dec 2008 at 11:35, HT wrote:

I can think of lots of arguments why this is a bad idea, but I don't
seem to be able to think of a really convincing one.


I think it depends on the problem domain.  As someone else said, there
are issues with being able to inject arbitrary code via the config file.
In some applications, this would be a feature, in others it would be a
security hole.

Another angle to look at is the audience for the config file.  If they
are all going to be python programmers or python-familiar, great.
If not...think about the user reaction to the tracebacks resulting from
typos.  If you use a purpose-designed config file (whether it is based on
ConfigParser or not), you can more easily generate helpful error messages.

--RDM
--
http://mail.python.org/mailman/listinfo/python-list

Re: simplest way to strip a comment from the end of a line?

[rdmurray]

On Thu, 4 Dec 2008 at 08:50, Joe Strout wrote:
I have lines in a config file which can end with a comment (delimited by # as 
in Python), but which may also contain string literals (delimited by double 
quotes).  A comment delimiter within a string literal doesn't count.  Is 
there any easy way to strip off such a comment, or do I need to use a loop to 
find each # and then count the quotation marks to its left?


>>> from shlex import split
>>> split("this is a test")
['this', 'is', 'a', 'test']
>>> split("this is a test #with a comment")
['this', 'is', 'a', 'test', '#with', 'a', 'comment']
>>> split("this is a test #with a comment", comments=True)
['this', 'is', 'a', 'test']
>>> split("this is a '#gnarlier' test #with a comment", comments=True)
['this', 'is', 'a', '#gnarlier', 'test']

http://docs.python.org/library/shlex.html

--RDM
--
http://mail.python.org/mailman/listinfo/python-list