Re: Can't define __call__ within __init__?

[Matt Nordhoff]

Neal Becker wrote:
> Simon Brunning wrote:
> 
>> On 10 March 2010 13:12, Neal Becker  wrote:
>>> Want to switch __call__ behavior.  Why doesn't this work?  What is the
>>> correct way to write this?
>>>
>>> class X (object):
>>> def __init__(self, i):
>>> if i == 0:
>>> def __call__ (self):
>>> return 0
>>> else:
>>> def __call_ (self):
>>> return 1
>>>
>>>
>>> x = X(0)
>>>
>>> x()
>>> TypeError: 'X' object is not callable
>> __call__ is in the __init__ method's local namespace - you need to
>> bind it to the class's namespace instead:
>>
>> X.__call__ = __call__
>>
>> But this probably isn't what you want either, since all instances of X
>> will share the same method.
>>
>> What are you trying to do? In your simple example, you'd be much
>> better off with a single __call__ method. But you knew that.
>>
> 
> Sorry, a bit early in the morning.  This works:
> class X (object):
> def __init__(self, i):
> if i == 0:
> def F (self):
> return 0
> else:
> def F (self):
> return 1
> self.F = F
> 
> def __call__ (self):
> return self.F (self)
>
> 
> Not sure if there is a more elegant (or compact) way to write this.
> Could __call__ be defined directly within __init__?
> 
> What I'm trying to do is make a callable whose behavior is switched based on 
> some criteria that will be fixed for all calls.  In my example, this will 
> ultimately be determined by the setting of a command line switch.

ISTM it would be prettiest to do:

class X(object):
def __init__(self, i):
self.flag = i == 0
def __call__(self):
if self.flag:
return 0
else:
return 1

Or, if the comparison isn't particularly expensive, it would look nicer
to just use self.i and do "self.i == 0" in __call__.

Not that it matters, but this is probably faster than your version, too,
since it saves a method call.

By the way, IIRC Python only looks up double-underscore methods on the
class, not the instance. That's why you had to indirect through self.F.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Six Minutes and fourty two seconds

[Matt Nordhoff]

Tobiah wrote:
> Now that I use python, this is the amount of time 
> per day that I spend adding forgotten semicolons while
> debugging other languages.

You can fix that by not using other languages. :>
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: getopt not raising exception

[Matt Nordhoff]

Matthew Lear wrote:
> Hello,
> 
> I'm having problems getting getopt to function correctly. Basically, no
> exception is being raised if no argument is passed to the code snippet
> below. I've read the Python documentation and tried example code from
> various sources which should cause an exception, only they don't. I've
> also tried executing the code on different machines too but to no avail.
> I'm sure I'm obviously doing something wrong but can't determine what.
> Any help would be much appreciated indeed.
> 
> import sys, getopt
> 
> try:
> opts, args = getopt.getopt(sys.argv, "h:", ["help"])
> except getopt.GetoptError:
> print "error"
> sys.exit(2)
> 
> If no args are passed when the script is run there is no exception
> raised. Why? Surely the "h:" means that this option must be passed?
> 
> Thanks,
> --  Matt

That's not what ":" means. It means that, *if* the option is passed, it
must be followed by an additional argument, e.g.:

foo.py -h something

If you require that -h be passed, it's not much of an option! You should
use a regular argument and check len(args) or somesuch. Or, if you must
keep it an "option", do something equivalent for that.

BTW: Checked out optparse? It's bigger and sexier!
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Another Sets Problem

[Matt Nordhoff]

Victor Subervi wrote:
> On Mon, Dec 28, 2009 at 1:41 PM, MRAB  <mailto:pyt...@mrabarnett.plus.com>> wrote:
>> DON'T USE BARE EXCEPTS!
>> 
>> (There are 2 in your code.)
> 
> There are times when they are *necessary*.

No, there aren't.

Even if there were, this is not one of those situations.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using time.gov to get the current time

[Matt Nordhoff]

Matt Nordhoff wrote:
> Rick wrote:
>> Is there any way to get the current time from time.gov using urllib2 or 
>> something similar?
>>
>> Does anyone have any examples of doing this?
>>
>> Thanks!
>> Rick King
>> Southfield MI
> 
> Why isn't your local system's clock accurate enough?

Bah, I'm better at IRC than e-mail... Anyway:

The easiest way to get highly-accurate time is with an "apt-get install
ntp" (or your OS's equivalent), and that solves it for all of your
applications, not just this one.

The time synchronization solutions that exist today are the result of
decades of development by experts. If you try to write some solution
yourself, it will not be as accurate (though that's okay if it still
meets your accuracy requirements), and you're liable to make some
mistake* and wind up with a bunch of angry NTP server operators chasing
after you with torches and pitchforks.

* Such as making excessively-frequent requests, or making more frequent
requests if the server appears down, or ignoring NTP Kiss-o'-Death
packets. Do not do these things!
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using time.gov to get the current time

[Matt Nordhoff]

Rick wrote:
> Is there any way to get the current time from time.gov using urllib2 or 
> something similar?
> 
> Does anyone have any examples of doing this?
> 
> Thanks!
> Rick King
> Southfield MI

Why isn't your local system's clock accurate enough?
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about dir function

[Matt Nordhoff]

Ray Holt wrote:
> When I run a dir(_builtins_) I get the error message that the name
> _builtins_ is not defined. I haven't tried the dir function on other
> functions, but can someone tell me why I am getting this message?
> Thanks, Ray

You are getting that message because the name "_builtins_" is not
defined, as it says. You were probably looking for "__builtins__", with
2 underscores on each end, not just 1.

BTW, "__builtins__" is a CPython implementation detail. If you want to
play with built-in objects, you should import the __builtin__ (no "s")
module and use that.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: a list/re problem

[Matt Nordhoff]

Grant Edwards wrote:
> On 2009-12-11, Ed Keith  wrote:
>> I have a problem and I am trying to find a solution to it that is both 
>> efficient and elegant.
>>
>> I have a list call it 'l':
>>
>> l = ['asc', '*nbh*', 'jlsdjfdk', 'ikjh', '*jkjsdfjasd*', 'rewr']
> 
>> Notice that some of the items in the list start and end with
>> an '*'. I wish to construct a new list, call it 'n' which is
>> all the members of l that start and end with '*', with the
>> '*'s removed.
>>
>> So in the case above n would be ['nbh', 'jkjsdfjasd']
> 
> [s[1:-1] for s in l if (s[0] == s[-1] == '*')]

s[0] and s[-1] raise an IndexError if l contains an empty string.

Better something like:

>>> [s[1:-1] for s in l if (s[:1] == s[-1:] == '*')]

Or just the slightly more verbose startswith/endswith version.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Variables with cross-module usage

[Matt Nordhoff]

Rami Chowdhury wrote:
> Hi Nitin,
> 
> On Sat, Nov 28, 2009 at 14:36, MRAB  wrote:
>> Nitin Changlani. wrote:
>>> three.py
>>> 
>>> import one
>>> import two
>>>
>>> def argFunc():
>>>one.x = 'place_no_x'
>>>one.a = 'place_no_a'
>>>one.b = 'place_no_b'
>>>
> 
> I think this is what is biting you. You might expect that after
> argFunc, one.x would be set to 'place_no_x' and so on. However,
> Python's scoping doesn't work like that -- the name one.x is only
> rebound in the function's scope, so outside of argFunc (e.g. in your
> main printing code) one.x is still bound to 'place_x'.
> 
> HTH,
> Rami

Not true. argFunc does not rebind the name "one", it mutates the module
object referred to by the name "one". Since there is only one instance
of a given module*, the change is indeed reflected everywhere the "one"
module is accessed.

The problem is that argFunc does not modify (or replace) one.myList, as
MRAB said.

* Unless you do something strange like reload() or editing sys.modules
or having module available under different names...or something.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Imitating "tail -f"

[Matt Nordhoff]

Jason Sewall wrote:
> FWIW, GNU tail on Linux uses inotify for tail -f:
> 
> http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/tail.c
> 
> The wikipedia page for inotify lists several python bindings:
> 
> http://en.wikipedia.org/wiki/Inotify
> 
> Not much help for non-Linux users, but there it is. Too bad, because
> inotify is pretty cool.
> 
> Jason

Some other operating systems have similar facilities, e.g. FSEvents on OS X.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: problem with read() write()

[Matt Nordhoff]

Gertjan Klein wrote:
> Alf P. Steinbach wrote:
> 
>> So with 'w+' the only way to get garbage is if 'read' reads beyond the end 
>> of 
>> file, or 'open' doesn't conform to the documentation.
> 
> It does read beyond the end of file. This is perhaps the way the
> underlying C library works, but it looks like an "unexpected feature"
> (read: bug) to me.
> 
> I reproduced (with Python 2.5.2 on WinXP) the code the OP wrote after
> creating an empty (0-byte) test file; after the write() the read()
> returns random garbage. I can't imagine why anyone would want that
> behaviour. The file grew to be 4099 bytes after f.close(). I wrote
> 'hello' to it, so the length of garbage added was 4094 bytes, which I
> find a strange number also.
> 
> I would have expected the read to return nothing. Can anyone explain or
> even defend this behaviour?
> 
> Gertjan.

I wonder, does it still happen if you open the file in binary mode?
(Stick a "b" in the file mode.) It could be some Windows text mode
craziness.
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: for loop: range() result has too many items

[Matt Nordhoff]

Matt Nordhoff wrote:
> Andre Engels wrote:
> [snip]
> 
>> However, I think that the better Python way would be to use a generator:
>>
>> def infinite_numbergenerator():
>> n = 0
>> while True:
>>  yield n
>>  n += 1
>>
>> for i in infinite_numbergenerator():
>> ...
> 
> That's what itertools.count() is for.
> 
> <http://docs.python.org/library/itertools.html#itertools.count>
> 
> It'll be faster, too, since it's written in C. Not that it really
> matters, but it's always nice.

...Although itertools.count() is limited by sys.maxint (well,
PY_SSIZE_T_MAX), so I guess it isn't useful in this case.
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: for loop: range() result has too many items

[Matt Nordhoff]

Andre Engels wrote:
[snip]

> However, I think that the better Python way would be to use a generator:
> 
> def infinite_numbergenerator():
> n = 0
> while True:
>  yield n
>  n += 1
> 
> for i in infinite_numbergenerator():
> ...

That's what itertools.count() is for.



It'll be faster, too, since it's written in C. Not that it really
matters, but it's always nice.
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: OT: Can;'t find a Mozilla user group

[Matt Nordhoff]

Anthra Norell wrote:
> I can't run Firefox and Thunderbird without getting these upgrade
> ordering windows. I don't touch them, because I have reason to suspect
> that they are some (Russian) virus that hijacks my traffic. Occasionally
> one of these window pops up the very moment I hit a key and next a
> confirmation message appears "reassuring" me that the upgrade will be
> installed the next time I start. I meant to ask Mozilla about their
> upgrade policy and facilities but for all the googling I do I can't find
> a contact address, nor an active user group. Hints will be greatly
> appreciated. Thanks!
> 
> Frederic

Mozilla software updates automatically by default, so that Joe User
won't run some out-of-date, insecure version.

Mozilla's newsgroups are available on , or you
can subscribe to the mailing list gateways at
.
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: confused with subprocess.Popen

[Matt Nordhoff]

Soumen banerjee wrote:
> Hello,
> for a certain app, i used to use a command:
> os.system("soundwrapper espeak -f line.txt")
> now, if i wanted to kill espeak, i would have to run:
> os.system("killall espeak")
> since the subprocess module allows sending SIGKILL to the process, i
> decided to switch to using it. However i cant pass the spaced out
> arguments to it. For example:
> a=subprocess.Popen("soundwrapper espeak -f line.txt")
> results in a OSError, no such file or directory.
> How do i use the Popen function?
> Regards
> Soumen

Read subprocess's documentation. It takes a list, not a string:

>>> a = subprocess.Popen(["soundwrapper", "espeak", "-f", "line.txt"])
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Simple way of handling errors

[Matt Nordhoff]

Steven D'Aprano wrote:
> On Wed, 06 May 2009 20:21:38 -0700, TomF wrote:
> 
>>> The only reason you would bother going to the time and effort of
>>> catching the error, printing your own error message, and then exiting,
>>> is if you explicitly want to hide the traceback from the user.
>> Well, to me, exposing the user to such raw backtraces is unprofessional,
>> which is why I try to catch user-caused errors.  But I suppose I have an
>> answer to my question.
> 
> That depends on your audience. Not every program is written to be used 
> for a technical incompetent audience. Some users actually *want* to see 
> the errors.
> 
> But certainly there are large classes of applications where you do want 
> to suppress the traceback. That's why I said "if you explicitly want to 
> hide the traceback from the user" rather than "don't do this".
> 
> The idiom I use is to wrap the *entire* application in a single 
> try...except block, and then put all your user-friendly error handling in 
> one place, instead of scattered over the entire application:
> 
> 
> try:
> main(sys.argv[1:])
> except KeyboardInterrupt, SystemExit:

That should be:

except (KeyboardInterrupt, SystemExit):

;-D

> raise
> except Exception, e:
> log(e)
> print >>sys.stderr, str(e)
> sys.exit(1)
> 
> 
> 
> Hope this helps.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Use of Unicode in Python 2.5 source code literals

[Matt Nordhoff]

Uncle Bruce wrote:
> I'm working with Python 2.5.4 and the NLTK (Natural Language
> Toolkit).  I'm an experienced programmer, but new to Python.
> 
> This question arose when I tried to create a literal in my source code
> for a Unicode codepoint greater than 255.  (I also posted this
> question in the NLTK discussion group).
> 
> The Python HELP (at least for version 2.5.4) states:
> 
> +++
> Python supports writing Unicode literals in any encoding, but you have
> to declare the encoding being used. This is done by including a
> special comment as either the first or second line of the source file:
> 
> #!/usr/bin/env python
> # -*- coding: latin-1 -*-
> 
> 
> Based on some experimenting I've done, I suspect that the support for
> Unicode literals in ANY encoding isn't really accurate.  What seems to
> happen is that there must be an 8-bit mapping between the set of
> Unicode literals and what can be used as literals.
> 
> Even when I set Options / General / Default Source Encoding to UTF-8,
> IDLE won't allow Unicode literals (e.g. characters copied and pasted
> from the Windows Character Map program) to be used, even in a quoted
> string, if they represent an ord value greater than 255.
> 
> I noticed, in researching this question, that Marc Andre Lemburg
> stated, back in 2001, "Since Python source code is defined to be
> ASCII..."
> 
> I'm writing code for linguistics (other than English), so I need
> access to lots more characters.  Most of the time, the characters come
> from files, so no problem.  But for some processing tasks, I simply
> must be able to use "real" Unicode literals in the source code.
> (Writing hex escape sequences in a complex regex would be a
> nightmare).
> 
> Was this taken care of in the switch from Python 2.X to 3.X?
> 
> Is there a way to use more than 255 Unicode characters in source code
> literals in Python 2.5.4?
> 
> Also, in the Windows version of Python, how can I tell if it was
> compiled to support 16 bits of Unicode or 32 bits of Unicode?
> 
> Bruce in Toronto

Works for me:

--- snip ---
$ cat snowman.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import unicodedata

snowman = u'☃'

print len(snowman)
print unicodedata.name(snowman)
$ python2.6 snowman.py
1
SNOWMAN
--- snip ---

What did you set the encoding to in the declaration at the top of the
file? The help text you quoted uses latin-1 as an example, an encoding
which, of course, only supports 256 code points. Did you try utf-8 instead?

The regular expression engine's Unicode support is a different question,
and I do not know the answer.

By the way, Python 2.x only supports using non-ASCII characters in
source code in string literals. Python 3 adds support for Unicode
identifiers (e.g. variable names, function argument names, etc.).
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: print(f) for files .. and is print % going away?

[Matt Nordhoff]

Esmail wrote:
> Hello all,
> 
> I use the print method with % for formatting my output to
> the console since I am quite familiar with printf from my
> C days, and I like it quite well.
> 
> I am wondering if there is a way to use print to write
> formatted output to files?
> 
> Also, it seems like I read that formatting with print is
> going away eventually and we should start using something
> else? Is that true and if so, what?
> 
> Thanks,
> Esmail

String formatting has nothing to do with the print statement/function.
It's an operator, just like doing "foo" + "bar"; you can use it wherever
you want.

Look:

>>> "%s" % ('foo',)
'foo'
>>> len("%s" % ('foo',))
3
>>> d = {}
>>> d["%s" % ('foo',)] = 1
>>> d
{'foo': 1}
>>> ("%s" % ('foo',)).upper()
'FOO'
>>>

See 
Also see  for
information on the replacement for the old string formatting, Python
2.6's new str.format().
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: JSON and Firefox sessionstore.js

[Matt Nordhoff]

Steven D'Aprano wrote:
> On Thu, 23 Apr 2009 05:08:35 +0100, I V wrote:
> 
>> For something with at least a vague air of credibility to it, somebody
>> who appears to be a Mozilla developer comments on their bug tracker,
>> that sessionstore.js isn't "pure JSON" (though he only identifies the
>> parantheses, which are much easier to fix, as being a problem):
>>
>> https://bugzilla.mozilla.org/show_bug.cgi?id=407110#c2
> 
> Ah, fantastic! Or rather, damn, I was hoping it was proper JSON and not 
> some sort of bizarre mutant.
> 
> For what it's worth, removing the leading/trailing parentheses gives a 
> different error:



> Presumably that's the error you get when you don't quote your property 
> names properly.

That bug is about changing it *to* JSON-plus-parentheses. The patch
landed [1] (and was subsequently backed out [2] and landed again [3]) in
time for Firefox 3.5, but unless you're running that, it's still
JavaScript source code, generated by toSource() [4]:


()

[1] 
[2] 
[3] 
[4]

()
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Scrap Posts

[Matt Nordhoff]

Avi wrote:
> Hey Folks,
> 
> I love this group and all the awesome and python savvy people who post
> here. However I also see some dumb posts like 'shoes' or something
> related to sex :(
> 
> What can we do about crap like this? Can we clean it up? Or atleast
> flag some for removal.
> 
> Moderators?

FWIW, this newsgroup is mirrored (both ways) as a mailing list. The list
server runs some pretty effective anti-spam software, and you could
subscribe to it:


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: safe eval of moderately simple math expressions

[Matt Nordhoff]

Joel Hedlund wrote:
> Hi all!
> 
> I'm writing a program that presents a lot of numbers to the user, and I
> want to let the user apply moderately simple arithmentics to these
> numbers. One possibility that comes to mind is to use the eval function,
> but since that sends up all kinds of warning flags in my head, I thought
> I'd put my idea out here first so you guys can tell me if I'm insane. :-)
> 
> This is the gist of it:
> --
> import math
> 
> globals = dict((s, getattr(math, s)) for s in dir(math) if '_' not in s)
> globals.update(__builtins__=None, divmod=divmod, round=round)
> 
> def calc(expr, x):
> if '_' in expr:
> raise ValueError("expr must not contain '_' characters")
> try:
> return eval(expr, globals, dict(x=x))
> except:
> raise ValueError("bad expr or x")
> 
> print calc('cos(x*pi)', 1.33)
> --
> 
> This lets the user do stuff like "exp(-0.01*x)" or "round(100*x)" but
> prevents malevolent stuff like "__import__('os').system('del *.*')" or
> "(t for t in (42).__class__.__base__.__subclasses__() if t.__name__ ==
> 'file').next()" from messing things up.
> 
> I assume there's lots of nasty and absolutely lethal stuff that I've
> missed, and I kindly request you show me the error of my ways.
> 
> Thank you for your time!
> /Joel Hedlund

I'm way too dumb and lazy to provide a working example, but someone
could work around the _ restriction by obfuscating them a bit, like this:

>>> '\x5f'
'_'
>>> getattr(42, '\x5f\x5fclass\x5f\x5f') # __class__


Is that enough to show you the error of your ways? :-D Cuz seriously,
it's a bad idea.

I'm sorry, but I don't know a good solution. The simplicity of eval is
definitely very attractive, but it's just not safe.

(BTW: What if a user tries to do some ridiculously large calculation to
DoS the app? Is that a problem?)
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting Binary content in files

[Matt Nordhoff]

ritu wrote:
> Hi,
> 
> I'm wondering if Python has a utility to detect binary content in
> files? Or if anyone has any ideas on how that can be accomplished? I
> haven't been able to find any useful information to accomplish this
> (my other option is to fire off a perl script from within m python
> script that will tell me whether the file is binary), so any pointers
> will be appreciated.
> 
> Thanks,
> Ritu

There isn't any perfect test. The usual heuristic is to check if there
are any NUL bytes in the file:

>>> '\0' in some_string

That can fail, of course. UTF-16-encoded text will have tons of NUL
bytes, and some binary files may not have any.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeEncodeError - opening encoded URLs

[Matt Nordhoff]

D4rko wrote:
> Hi!
> 
> I have a problem with urllib2 open() function. My application is
> receiving the following request - as I can see in the developement
> server console it is properly encoded:
> 
> [27/Mar/2009 22:22:29] "GET /[blahblah]/Europa_%C5%9Arodkowa/5 HTTP/
> 1.1" 500 54572
> 
> Then it uses this request parameter as name variable to build
> wikipedia link, and tries to acces it with following code:
> 
>   url = u'http://pl.wikipedia.org/w/index.php?title=' + name +
> '&printable=yes'
>   opener = urllib2.build_opener()
>   opener.addheaders = [('User-agent', 'Mozilla/5.0')]
>   wikipage = opener.open(url)
> 
> Unfortunately, the last line fails with the exception:
> UnicodeEncodeError 'ascii' codec can't encode character u'\u015a' in
> position 30: ordinal not in range(128).  Using urlencode(url) results
> in TypeError "not a valid non-string sequence or mapping object", and
> quote(url)  fails because of KeyError u'\u015a' . How can I properly
> parse this request to make it work (ie. acces
> http://pl.wikipedia.org/wiki/Europa_%C5%9Arodkowa)?

What if you just used a regular byte string for the URL?

>>> url = 'http://pl.wikipedia.org/w/index.php?title=' + name +
'&printable=yes'

(Unless "name" is a unicode object as well.)

(Nice user-agent, BTW. :-P )
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: unsubscribe to send daily mails

[Matt Nordhoff]

Sudheer Rapolu wrote:
> Hello
> 
> Please unsubscribe to send daily mails to me.
> 
> Warm Regards
> Sudheer
> 
> 
> 
> 
> --
> http://mail.python.org/mailman/listinfo/python-list

See the link in the signature of every message, or the headers of every
message:

> List-Unsubscribe: ,
>   
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: download x bytes at a time over network

[Matt Nordhoff]

Saurabh wrote:
> Heres the reason behind wanting to get chunks at a time.
> Im actually retrieving data from a list of RSS Feeds and need to
> continuously check for latest posts.
> But I dont want to depend on Last-Modified header or the pubDate tag
> in . Because a lot of feeds just output date('now')  instead
> of the actual last-updated timestamp.
> But when continuously checking for latest posts, I dont want to
> bombard other people's bandwidth - so I just want to get chunks of
> bytes at a time and internally check for ... with my
> database against timestamp values.
> Is there a better way to achieve this ?

For the feeds that *do* set Last-Modified properly, won't you be using
*more* bandwidth by downloading part of the feed instead of just using
If-Modified-Since?
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: print - bug or feature - concatenated format strings in a print statement

[Matt Nordhoff]

bdb112 wrote:
> Thanks for all the replies:
> I think I see now - % is a binary operator whose precedence rules are
> shared with the modulo operator regardless of the nature of its
> arguments, for language consistency.
> I understand the arguments behind the format method, but hope that the
> slightly idiosyncratic print( ..% ..) remains, as the vaguely
> pictorial "printf" format string is clearer for a long line with
> several arguments.
> I will use the "implicit string concatenation" to solve my problem but
> it is a little odd that an "invisible" operator is stronger than a
> visible one. (+).

The implicit string concatenation is actually done by the compiler; it
isn't an operator at all. Look:

>>> import dis
>>> def f():
... return "foo" "bar"
...
>>> dis.dis(f)
  2   0 LOAD_CONST   1 ('foobar')
  3 RETURN_VALUE
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Set & Frozenset?

[Matt Nordhoff]

Alan G Isaac wrote:
>> Hans Larsen schrieb:
>>> How could I "take" an elemment from a set or a frozenset 
> 
> 
> On 3/8/2009 2:06 PM Diez B. Roggisch apparently wrote:
>> You iterate over them. If you only want one value, use
>> iter(the_set).next()
> 
> 
> I recall a claim that
> 
> for result in myset: break
> 
> is the most efficient way to get one result.
> Is this right? (It seems nearly the same.)
> 
> Alan Isaac

Checking Python 2.5 on Linux, your solution is much faster, but seeing
as they both come in under a microsecond, it hardly matters.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: A tale of two execs

[Matt Nordhoff]

aha wrote:
> Hello All,
>   I am working on a project where I need to support versions of Python
> as old as 2.3. Previously, we distributed Python with our product, but
> this seemed a bit silly so we are no longer doing this.  The problem
> that I am faced with is that we have Python scripts that use the
> subprocess module, and subprocess is not available in Python 2.3.
> Below is the strategy I am thinking about using, however if, you have
> better ideas please let me know.
> 
> def runner(cmd, stdin, stdout, ...):
>   try:
> import subprocess
> sbm = 1
>   except:
> sbm = 0
> 
>   # Now do something
>   if sbm:
> process = subporcess(...)
>   else:
> import popen2
> process = popen2.Popen4(...)
> 
> Has anyone else run into a situation similar to this one?

FWIW, the Mercurial project went the other way: They wrote "popen2",
"popen3" and "Popen3" functions that were wrappers around subprocess:


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Revision Control

[Matt Nordhoff]

Tim Chase wrote:



>> -Mercurial (this is a big up and coming RCS)
> 
> This is currently my favorite:  good branching/merging, fast, written
> mostly in Python (one C extension module, IIRC), and a simple interface
> 
>> -Bazaar (written in Python. Also pretty new. I don't know about Windows
>> support)
> 
> I like Bazaar (bzr), but the last several times I've tried it, it's been
> a little slow.  This has been steadily improving, and I like their
> emphasis on correctness foremost.  It's a lot like mercurial, but is
> pure python (no C extension module) which makes it nice to deploy into
> environments such as my shared web-hosting service where I can't build
> extension C modules or install packages such as hg/git/svn from the
> repository.

FWIW, Bazaar and Mercurial both have about half a dozen C modules. (Most
of Bazaar's are Pyrex, though, not straight C.)

The next release of Mercurial will add support for running without them.
 (Bazaar already supports that, of course.)


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: time: Daylight savings confusion

[Matt Nordhoff]

Tim H wrote:
> On Win XP 64bit, Python 2.6.1 64bit
> 
> I am trying to rename files by their creation time.
> 
> It seems the time module is too smart for its own good here.
> 
> time.localtime(os.path.getctime(f)) returns a value one hour off from
> what windows reports for files that were created when Daylight savings
> time was in effect (ie when the tm_isdst field is one).  I can kludge
> it, but am I missing the "right" way to do it?
> 
> Tim
> 
> full code:



>From what I remember, this is a bug in the Win32 API. Python probably
could hack around it, but, well, I guess it doesn't.

Googling a bit, Perl's Win32::UTCFileTime module [1] provides
workarounds (not for Python, of course, but it could probably be ported)
and explains all the gory details (which I have not read, so maybe I
misunderstand ;-).

[1]


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: v = json.loads("{'test':'test'}")

[Matt Nordhoff]

Matt Nordhoff wrote:
> gert wrote:
>> On Jan 25, 11:16 pm, Дамјан Георгиевски  wrote:
>>>> raise ValueError(errmsg("Expecting property name", s, end))
>>>> http://docs.python.org/library/json.html
>>>> What am I doing wrong ?
>>> try this
>>> v = json.loads('{"test":"test"}')
>>>
>>> JSON doesn't support single quotes, only double quotes.
>> the funny part is when you print(v) you get
>> {'test': 'test'}
>>
>> Single quotes works in every browser that support json so i
>> recommended python should support it too, besides it looks much
>> cleaner
>> {'test': 'test'}
>> {"test": "test"}
>>
>> It can not be that hard to support both notation can it ?
> 
> There's a difference between JavaScript source code and JSON. AFAICT
> from the source [1], Mozilla's JSON parser doesn't accept single quotes.
> 
> [1] <http://mxr.mozilla.org/mozilla-central/source/js/src/json.cpp>

By the way, I forgot to add, according to the ECMA-262 standard (page
18, section 7.8.4) [1], ECMAScript string literals can use either double
or single quotes, so that's not a browser-specific extension.

[1]
<http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf>
(<http://xrl.us/bedr3c>)
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: v = json.loads("{'test':'test'}")

[Matt Nordhoff]

gert wrote:
> On Jan 25, 11:16 pm, Дамјан Георгиевски  wrote:
>>> raise ValueError(errmsg("Expecting property name", s, end))
>>> http://docs.python.org/library/json.html
>>> What am I doing wrong ?
>> try this
>> v = json.loads('{"test":"test"}')
>>
>> JSON doesn't support single quotes, only double quotes.
> 
> the funny part is when you print(v) you get
> {'test': 'test'}
> 
> Single quotes works in every browser that support json so i
> recommended python should support it too, besides it looks much
> cleaner
> {'test': 'test'}
> {"test": "test"}
> 
> It can not be that hard to support both notation can it ?

There's a difference between JavaScript source code and JSON. AFAICT
from the source [1], Mozilla's JSON parser doesn't accept single quotes.

[1] 
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: using subprocess module in Python CGI

[Matt Nordhoff]

ANURAG BAGARIA wrote:
> Hello,
> 
> I am a Python Newbie and would like to call a short python script via
> browser using a CGI script, but initially I am trying to call the same
> python script directly through python command line. The script intends
> to perform a few command line in a pipe and I have written the script (a
> short one) as follows.
> 
> #!/usr/bin/python
> 
> import cgi, string, os, sys, cgitb, commands, subprocess
> import posixpath, macpath
> #file = "x.tar.gz"
> #comd = "tar -xf %s" % (file)
> #os.system(comd)
> #commands.getoutput('tar -xf x.tar.gz | cd demo; cp README ../')
> comd = [\
> "tar -xf x.tar.gz", \
> "cd demo", \
> "cp README ../", \
>   ]

That's not how subprocess.call() works. You're trying to run an
executable called "tar -xf x.tar.gz", passing it the arguments "cd demo"
and "cp README ../".

> outFile = os.path.join(os.curdir, "output.log")
> outptr = file(outFile, "w")
> errFile = os.path.join(os.curdir, "error.log")
> errptr = file(errFile, "w")
> retval = subprocess.call(comd, 0, None, None, outptr, errptr)
> errptr.close()
> outptr.close()
> if not retval == 0:
> errptr = file(errFile, "r")
> errData = errptr.read()
> errptr.close()
> raise Exception("Error executing command: " + repr(errData))
> 
> 
> but after trying to execute this independently, I get the following
> error which I am unable to interpret :
> 
> Traceback (most recent call last):
>   File "process.py", line 18, in 
> retval = subprocess.call(comd, 0, None, None, outptr, errptr)
>   File "/usr/lib/python2.5/subprocess.py", line 443, in call
> return Popen(*popenargs, **kwargs).wait()
>   File "/usr/lib/python2.5/subprocess.py", line 593, in __init__
> errread, errwrite)
>   File "/usr/lib/python2.5/subprocess.py", line 1135, in _execute_child
> raise child_exception
> 
> 
> Could someone suggest where am I going wrong and if corrected, what is
> the probability of this script being compatible with being called
> through the browser. Thanking you people in advance.

Well, you'd need to output something, but otherwise, sure, why not?

print "Content-Type: text/html"
print
print "..."

> Regards.

Why do you even need to use subprocess to do this? All it's doing is
extracting the README file from a tarball, right? You can use the
tarfile module for that.


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Free place to host python files?

[Matt Nordhoff]

James Mills wrote:
> On Wed, Dec 17, 2008 at 10:25 AM, Chris Rebert  wrote:
>> I'll plug Bitbucket (http://bitbucket.org/). It gives you 150MB of
>> Mercurial hosting for free, along with a bug tracker and wiki. And I
>> hear it's implemented using Django.
> 
> FreeHG (http://freehg.org) is pretty good too :)
> 
> cheers
> James

I'll throw in  too. No wiki, though.

And , I guess.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using the `email' module in a bazaar plugin

[Matt Nordhoff]

Julian Smith wrote:
> I don't seem to be able to use the `email' module from within a bazaar
> plugin. Other modules work ok, e.g. nntplib.
> 
> Here's my plugin, cut-down to show just the email problem:
> 
>   import sys
>   print 'sys.version', sys.version
> 
>   import nntplib
>   n = nntplib.NNTP( 'jsmith-ubuntu2' )
>   print n
> 
>   import email
>   m=email.MIMEText.MIMEText('text of email')
> 
> - and here's the output when i run any bazaar command:
> 
>   > bzr status
>   sys.version 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
>   [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)]
>   
>   'module' object has no attribute 'MIMEText'
>   Unable to load plugin 'js_notify' from '/home/jsmith/.bazaar/plugins'
>   ...
> 
> The plugin runs fine on its own, it's only when loaded into Bazaar that things
> go wrong.
> 
> I thought perhaps this could be caused by the `email' module's use of
> LazyImporter. I've tried various things such as `from email.mime.text import
> MIMEText', and they fail in similar ways.
> 
> I'm using bzr-1.9, python 2.5.2, on Ubuntu-8.xx.
> 
> Does anyone have any ideas ?
> 
> Thanks,
> 
> - Julian

The built-in email module is probably getting shadowed by another one
(perhaps bzrlib.plugins.email?), so "import email" is importing the
wrong module. You'll have to rename it to something else.

Have your plugin print email.__path__ to see which module it is.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: html codes

[Matt Nordhoff]

Daniel Fetchinson wrote:
> Hi folks,
> 
> I came across a javascript library that returns all sorts of html
> codes in the cookies it sets and I need my web framework (written in
> python :)) to decode them. I'm aware of htmlentitydefs but
> htmlentitydefs.entitydefs.keys( ) are of the form '&#xxx' but this
> javascript library uses stuff like '%3A' for the ':' for example. The
> conversion is here:
> 
> http://www.ascii.cl/htmlcodes.htm
> 
> Is there a python package/module/whatever that does the conversion for
> me or do I have to write a little wrapper myself (and introduce bugs
> while doing so :))?
> 
> Cheers,
> Daniel

>>> import urllib
>>> urllib.unquote('%20')
' '


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: generating a liste of characters

[Matt Nordhoff]

This is a slightly old post, but oh well...

Shane Geiger wrote:
> import string
> alphabet=list(string.letters[0:26])
> print alphabet

Most of the string module's constants are locale-specific. If you want
the ASCII alphabet instead of the current language's, you need to use
string.ascii_{letters,lowercase,uppercase}.

> Yves Dorfsman wrote:
>> Is there any built in way to generate a list of characters, something
>> along the line of range('a'-'z') ?
>>
>> Right now I am using:
>>
>>   chars  = [ chr(l)  for l in range(0x30, 0x3a) ] # 0 - 9
>>   chars += [ chr(l)  for l in range(0x41, 0x5b) ] # A - Z
>>   chars += [ chr(l)  for l in range(0x61, 0x7b) ] # a - z
>>
>> Is there a better, more straight forward way of doing that ?
>>
>>
>>
>> Thanks.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Need help in understanding a python code

[Matt Nordhoff]

Benjamin Kaplan wrote:
> If you really believe that, you haven't been following this list long
> enough. Every terminology dispute always includes at least 1 Wikipedia
> link.
> 
> Also, you might want to look at this study:
> http://news.cnet.com/2100-1038_3-5997332.html

That study has been disputed; see the links at the top of
.

/me ducks
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Simple print to stderr

[Matt Nordhoff]

RC wrote:
> By default the print statement sends to stdout
> I want to send to stderr
> 
> Try
> 
> print "my meeage", file=sys.stderr
> 
> I got
>> SyntaxError: invalid syntax
> 
> I try
> 
> print "my message", sys.stderr
> 
> But it still sent to stdout.
> What is the syntax?
> 
> I wouldn't understand Python's manual



That's only in Python 3 (or 2.6 with the proper __future__ import).
Before that, print is a statement. You'd do it like this:

print >> sys.stderr, "whatever"

You should look at
,
not Python 3's documentation.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Antigravity module needed.

[Matt Nordhoff]

Terry Reedy wrote:
> If Python added an antigravity module to the stdlib,
> what should it say or do?  See
> http://xkcd.com/353/
> (and let your mouse hover).

It was added 2 days ago. :-P


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing my own Python interpreter

[Matt Nordhoff]

Ognjen Bezanov wrote:
> Hello All,
> 
> I am a third year computer science student and I'm the process of
> selection for my final year project.
> 
> One option that was thought up was the idea of implement my own version
> of the python interpreter (I'm referring to CPython here). Either as a
> process running on another OS or as a process running directly on the CPU.
> 
> Now, I can't seem to find a decent source of information on the python
> interpreter. I have made the assumption that Python works very much like
> Java, you have code that is compiled into bytecode, which is then
> executed in a virtual machine. IS this correct? Is there a good source
> to give me an overview of Python internals? (I can look at the code, but
> I would find it easier to understand if I can see the "big picture" as
> well)
> 
> Also, any pro's out there willing to chime on the feasibility of
> implementing python to run directly on the hardware (without an
> underlying OS)? I don't expect 100% compatibility, but would the basics
> (branching, looping, arithmatic) be feasible?
> 
> Thank you,
> 
> 
> Ognjen

FWIW... There are several other implementations of Python:

IronPython (.Net)
Jython (Java)
PyPy (Python) 

You might find working on one of them interesting, or maybe even CPython
itself.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Inefficient summing

[Matt Nordhoff]

Chris Rebert wrote:
> I personally would probably do:
> 
> from collections import defaultdict
> 
> label2sum = defaultdict(lambda: 0)

FWIW, you can just use:

label2sum = defaultdict(int)

You don't need a lambda.

> for r in rec:
> for key, value in r.iteritems():
> label2sum[key] += value
> 
> ratio = label2sum["F1"] / label2sum["F2"]
> 
> This iterates through each 'r' only once, and (imho) is pretty
> readable provided you know how defaultdicts work. Not everything has
> to unnecessarily be made a one-liner. Coding is about readability
> first, optimization second. And optimized code should not be
> abbreviated, which would make it even harder to understand.
> 
> I probably would have gone with your second solution if performance
> was no object.
> 
> Cheers,
> Chris
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Are spams on comp.lang.python a major nuisance?

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> I took over spam filter management for the python.org mailing lists a couple
> months ago and made a few changes to the way the spam filter is trained.
> Things seem to be at a reasonable level as far as I can tell (I see a few
> spams leak through each day), though I wasn't actively reading
> comp.lang.python/python-list@python.org before I took over the task, so I
> have nothing to compare with.  Does the level of spam leaking through the
> filter now seem excessive?  Is it more or less than in June and July?
> 
> Thanks,
> 
> Skip Montanaro

Over the last 24 hours, I think I count 10 spams, but that's probably
higher than usual.

I don't know if it's "excessive". It certainly isn't ideal, but it only
takes 30 seconds to get rid of them, and we need to give my spam filter
something to do, right? :-)

I haven't been reading the list for very long, so I don't know how it
compares to June or July. It feels like it's been increasing over time,
but I don't have any numbers to back that up.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to parse a string completely into a list

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> On Sep 24, 9:44 pm, "Chris Rebert" <[EMAIL PROTECTED]> wrote:
>> On Wed, Sep 24, 2008 at 8:30 PM,  <[EMAIL PROTECTED]> wrote:
>>> I want to take a long alpha-numeric string with \n and white-space and
>>> place ALL elements of the string (even individual parts of a long
>>> white-space) into separate list elements. The most common way I've
>>> seen this performed is with the split() function, however I don't
>>> believe that it has the power to do what I am looking for.
>>> Any suggestions?
>>> thanks
>> Could you please define exactly what you mean by "elements" of a string?
>>
>> If you mean characters, then just use list():>>> list("  \n \t abc")
>>
>> [' ', ' ', '\n', ' ', '\t', ' ', 'a', 'b', 'c']
>>
>> Regards,
>> Chris
> 
> Worked like a charm.
> kudos!

Why do you need to convert it to a list? Strings are sequences, so you
can do things like slice them or iterate through them by character:

>>> for character in "foo":
... print character
...
f
o
o
>>>
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Schwartzian transform for tuple in list

[Matt Nordhoff]

Chris Rebert wrote:
> On Wed, Sep 24, 2008 at 2:02 PM, David Di Biase <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> I have a rather large list structure with tuples contained in them (it's
>> part of a specification I received) looks like so:
>> [(x1,y1,r1,d1),(x2,y2,r2,d2)...]
>>
>> The list can range from about 800-1500 tuples in size and I'm currently
>> sorting it with this:
>>
>> a_list.sort(lambda a, b: cmp(b[3], a[3]))
> 
> You'd probably be better off using the 'key' keyword argument to
> .sort(), which is made for the Schwartzian Transform:
> 
> a_list.sort(key=lambda x: x[3])
> 
> This sorts the list items by the value produced by the key function
> for each item. It's also (IIRC) faster than using a comparison
> function like you're currently doing.
> 
> Regards,
> Chris

Yes, using 'key' is faster than 'cmp'.

If you have Python 2.4 or newer, it would also be slightly faster to use
operator.itemgetter() instead of a lambda:

>>> import operator
>>> a_list.sort(key=operator.itemgetter(3))

>> I'm actually sorting it by the last value in the tuple (d2). I have been
>> researching more efficient sorting algorithms and came across Schwartzian
>> transform via these links:
>>
>> http://www.biais.org/blog/index.php/2007/01/28/23-python-sorting-efficiency
>> http://dev.fyicenter.com/Interview-Questions/Python/Can_you_do_a_Schwartzian_Transform_in_Python_.html
>>
>> I get what's happening (sorta...errr...lol)  but I'm not sure if it is more
>> efficient in my scenario, if it is then I have no idea how to implement it
>> properly :-/
>>
>> Would be great if a true expert would offer a suggestion for me...
>>
>> Thanks!
>>
>> David
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Gateway to python-list is generating bounce messages.

[Matt Nordhoff]

Grant Edwards wrote:
> On 2008-09-12, Matt Nordhoff <[EMAIL PROTECTED]> wrote:
> 
>> I think you misunderstand. He's referring to the Sender
>> header, not the From header. The messages the listbot sends
>> out have a Sender header of
>> "[EMAIL PROTECTED]" (supposing
>> the subscriber's email address is [EMAIL PROTECTED]). Bounces
>> should be directed to the bitbucket or list admin or whatever,
>> not the user in the From header. kring.com just has a broken
>> mail server.
> 
> So the statement below by Dennis Lee Bieber in message
> [EMAIL PROTECTED] isn't actually
> correct?
> 
>   >>Three: The bounce/ooo-reply is sent to the message author, not
>   >>to any intermediate host(s). After all, on that end, it's
>   >>normal email failure response -- notify the author of the
>   >>message. It doesn't matter that the original message was posted
>   >>on a Usenet newsgroup if that group is automatically relayed to
>   >>members of a mailing list.

I have no idea. My post was assuming that Sjoerd Mullender was correct.

With headers like "Return-Path", "Errors-To" and "Sender", ISTM the
mailing list software is doing everything it can to avoid bounces
getting sent to the user in the From header. If it's completely wrong,
then, well, it would be silly to have gone to the effort.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Gateway to python-list is generating bounce messages.

[Matt Nordhoff]

Steven D'Aprano wrote:
> On Thu, 11 Sep 2008 17:27:33 +0200, Sjoerd Mullender wrote:
> 
>> When mail messages bounce, the MTA (Message Transfer Agent--the program
>> that handles mail) *should* send the bounce message to whatever is in
>> the Sender header, and only if that header does not exist, should it use
>> the From header.
> 
> Who makes up these rules, and why should we pay the least bit of 
> attention to them?
> 
> It's one thing to say "right or wrong, that's what list admins do and you 
> have to deal with their behaviour whatever way you can". It's another 
> thing altogether to take the legalistic attitude of "never mind the 
> consequences, the standard is the standard and must be unthinkingly 
> obeyed". If the standard does more harm than good, then ignoring the 
> standard is the right thing to do. (Better would be to change the 
> standard, but that probably won't happen until there's a critical mass of 
> people who ignore the existing broken standard and form their own de 
> facto standard.)
> 
> A standard isn't "correct" just because it's a standard, it's merely 
> something that a committee has agreed to do. In other words, it's a 
> compromise. Now, such compromises might be good and useful, or they might 
> combine the worst of all opinions. Just because something is standardized 
> doesn't make it the right thing to do. If you want proof of this, I give 
> you the recently approved ISO standard for Microsoft's so-called "Office 
> Open XML" OOXML file format.
> 
> The standard behaviour of sending bounce and out-of-office messages to 
> the sender works well when sending email to individuals, but for mailing 
> lists it is pointless and counter-productive. Pointless, because the 
> sender can't do anything to fix the problem he's being notified about. 
> And counter-productive, because it is an anti-feature, something that 
> makes the mailing list more unpleasant and less useful. Anyone who has 
> regularly emailed to a large mailing list has surely experienced the 
> frustration of receiving bounce messages from perfect strangers.
> 
> To anyone who wishes to defend the process of sending mailing list 
> bounces back the sender, ask yourself this: what do YOU do with such 
> bounces when you receive them? If you ignore them or delete them (whether 
> manually or via a procmail recipe or some other automatic system) then 
> what benefit does the standard behaviour offer?

I think you misunderstand. He's referring to the Sender header, not the
>From header. The messages the listbot sends out have a Sender header of
"[EMAIL PROTECTED]" (supposing the
subscriber's email address is [EMAIL PROTECTED]). Bounces should be
directed to the bitbucket or list admin or whatever, not the user in the
>From header. kring.com just has a broken mail server.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: manipulating files within 'for'

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> Ben Keshet:
>> ...wrong.  I thought I should omit the comma and didn't put it.  I guess
>> that stating the obvious should be the first attempt with beginners like
>> me.  Thanks for thinking about it (it's running perfect now).
> 
> In CLisp, Scheme etc, lists such commas aren't necessary, but in
> Python if you don't separate strings with a comma they become merged
> into a single string:
> 
 'foo', 'bar'
> ('foo', 'bar')
 'foo' 'bar'
> 'foobar'
> 
> Your mistake is caused by Python not following one of its general
> rules:
> 
> Explicit is better than implicit.
> 
> In such case the string concatenation (+) is done implicitly. It's a
> little handy feature once in a while (but not for me so far), while it
> may cause bugs, so I don't like this little feature of Python and I
> may like to see it removed, because it may bite you in similar
> situations, where you forgot a comma for mistake:
> 
> parts = ["foo", "bar" "baz"]
> 
> Bye,
> bearophile

It's useful when wrapping a line. For lack of better lorem ipsum:

whatever = some_function("Your mistake is caused by Python not "
 "following one of its general rules:\n\n"
 "Explicit is better than implicit.")

You can also use backslashes, and probably even + if you want to, but
the implicit concatenation is prettier (IMO, at least ;-).

But you do have a point. I have never thought about the problems it
could cause.

BTW, I could easily be wrong, but I think C behaves the same way as Python.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: function return

[Matt Nordhoff]

Gabriel Genellina wrote:
> En Thu, 11 Sep 2008 10:59:10 -0300, Matt Nordhoff
> <[EMAIL PROTECTED]> escribió:
> 
>>>result = "%s\t%s\t%s" %(id,gene_symbol,ptms)
>>
>> This is very trivial, but you could change the above line to:
>>
>>result = "\t".join(id, gene_symbol, ptms)
> 
> So trivial that you did not even attempt to test it, I presume. It must
> obviously work... except it doesn't :)
> 
> result = "\t".join((id, gene_symbol, ptms))

Haha, whoops. Sorry!
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: function return

[Matt Nordhoff]

Beema Shafreen wrote:
> hi all,
> 
> I have a script using functions , I have a problem in returning the
> result. My script returns only one line , i donot know where the looping
> is giving problem, Can any one suggest, why this is happening and let me
> know how to return all the lines
> 
> def get_ptm():
> fh = open('file.txt','r')
>data_lis = []
>for line in fh.readlines():

As an aside, you should change the above line to:

   for line in fh:

"readlines()" will read the entire file into memory at once, while just
iterating through it won't, so you'll save memory on large files.

>data = line.strip().split('\t')
>id = data[0].strip()
>gene_symbol = data[1].strip()
>ptms = data[8].strip()
>result = "%s\t%s\t%s" %(id,gene_symbol,ptms)

This is very trivial, but you could change the above line to:

   result = "\t".join(id, gene_symbol, ptms)

>return result
>  fh.close()
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: check if the values are prensent in a list of values

[Matt Nordhoff]

Emile van Sebille wrote:
> flit wrote:
>> Hello All,
>>
>> I will appreciate the help from the more skillfull pythonistas..
>>
>> I have a small app that generates a sequence like
>>
>> 00341
>> 01741
>> 03254
> 
> Consider using a dict with sorted tuple keys, eg
> 
> d = {}
> 
> for seq in ['00341','01741','03254']:
> ky = list(seq)
> ky.sort()
> d[tuple(ky)] = None
> 
> 
> then d.keys() are the unique combinations.
> 
> HTH,
> 
> Emile

I'm not judging whether this is a good solution or not, but that's a
silly use of a dict. A set would be better.

s = set()
for seq in ['00341','01741','03254']:
s.add(tuple(sorted(ky)))

Then you just, well, access the set directly, instead of using d.keys()
or something.

(I also replaced the sorting with the sorted() function for brevity.
This all assumes you have at least Python 2.4...)

>> This values I am putting in a list.
>>
>> So I have a list = [00341,01741,03254]
>>
>> after the programs find the sequence 03401 this sequence is "new" so
>> it appends on the list. But I want to avoid that as the   values are
>> already on the first sequence of the list (00341).
>> If I try to  use a "in" statement it will give false. as 00341 is
>> different from 00341 (but for my goal not..)
>>
>>
>> How can I check against this list and avoid to put "different"
>> sequences but same values?
>>
>> as 34100 --> dont append on the list
>> 14300 ---> dont append on the list
>> 05321 --> append to the list.
>>
>> Am I doing some conceptual error using lists?
>> There is a better approach?
>>
>> Thanks
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: "AttributeError: 'module' object has no attribute 'getdefaultlocale'" on Python start

[Matt Nordhoff]

Barak, Ron wrote:
> Hi Fellow Pythonians,
>  
> I stated getting the following when starting Python (2.5.2 on Windows XP):
>  
> C:\Documents and Settings\RBARAK>python -v
> # installing zipimport hook
> import zipimport # builtin
> # installed zipimport hook
> # D:\Python25\lib\site.pyc matches D:\Python25\lib\site.py
> import site # precompiled from D:\Python25\lib\site.pyc
> # D:\Python25\lib\os.pyc matches D:\Python25\lib\os.py
> import os # precompiled from D:\Python25\lib\os.pyc
> import errno # builtin
> import nt # builtin
> # D:\Python25\lib\ntpath.pyc matches D:\Python25\lib\ntpath.py
> import ntpath # precompiled from D:\Python25\lib\ntpath.pyc
> # D:\Python25\lib\stat.pyc matches D:\Python25\lib\stat.py
> import stat # precompiled from D:\Python25\lib\stat.pyc
> # D:\Python25\lib\UserDict.pyc matches D:\Python25\lib\UserDict.py
> import UserDict # precompiled from D:\Python25\lib\UserDict.pyc
> # D:\Python25\lib\copy_reg.pyc matches D:\Python25\lib\copy_reg.py
> import copy_reg # precompiled from D:\Python25\lib\copy_reg.pyc
> # D:\Python25\lib\types.pyc matches D:\Python25\lib\types.py
> import types # precompiled from D:\Python25\lib\types.pyc
> import _types # builtin
> import locale # directory
> D:\Python25\Lib\site-packages\wx-2.8-msw-unicode\wx\locale
> import locale # precompiled from
> D:\Python25\Lib\site-packages\wx-2.8-msw-unicode\wx\locale\__init__.pyc

For some reason, when you run "import locale", it's importing wx.locale
instead of the "locale" module from the stdlib. However, I have no idea
why that would be happening... Did you do something to add wx to your
PYTHONPATH? Maybe do "import sys; print sys.path" or "import os; print
os.environ['PYTHONPATH']" to verify this.

(Hopefully someone more knowledgeable than me will come along soon.)

> # D:\Python25\lib\codecs.pyc matches D:\Python25\lib\codecs.py
> import codecs # precompiled from D:\Python25\lib\codecs.pyc
> import _codecs # builtin
> import encodings # directory D:\Python25\lib\encodings
> # D:\Python25\lib\encodings\__init__.pyc matches
> D:\Python25\lib\encodings\__init__.py
> import encodings # precompiled from D:\Python25\lib\encodings\__init__.pyc
> # D:\Python25\lib\encodings\aliases.pyc matches
> D:\Python25\lib\encodings\aliases.py
> import encodings.aliases # precompiled from
> D:\Python25\lib\encodings\aliases.pyc
> 'import site' failed; traceback:
> Traceback (most recent call last):
>   File "d:\Python25\lib\site.py", line 415, in 
> main()
>   File "d:\Python25\lib\site.py", line 406, in main
> aliasmbcs()
>   File "d:\Python25\lib\site.py", line 357, in aliasmbcs
> enc = locale.getdefaultlocale()[1]
> AttributeError: 'module' object has no attribute 'getdefaultlocale'
> # D:\Python25\lib\warnings.pyc matches D:\Python25\lib\warnings.py
> import warnings # precompiled from D:\Python25\lib\warnings.pyc
> # D:\Python25\lib\linecache.pyc matches D:\Python25\lib\linecache.py
> import linecache # precompiled from D:\Python25\lib\linecache.pyc
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
> (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.

>  
> Has anyone else gotten this error ?
> Could anyone suggest a solution ?
>  
> Thanks,
> Ron.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: compare unicode to non-unicode strings

[Matt Nordhoff]

Asterix wrote:
> how could I test that those 2 strings are the same:
> 
> 'séd' (repr is 's\\xc3\\xa9d')
> 
> u'séd' (repr is u's\\xe9d')

You may also want to look at unicodedata.normalize(). For example, é can
be represented multiple ways:

>>> import unicodedata
>>> unicodedata.normalize('NFC', u'é')
u'\xe9'
>>> unicodedata.normalize('NFD', u'é')
u'e\u0301'
>>> u'\xe9' == u'e\u0301'
False

The first form is "composed", just being U+00E9 (LATIN SMALL LETTER E
WITH ACUTE). The second form is "decomposed", being made up of U+0065
(LATIN SMALL LETTER E) and U+0301 (COMBINING ACUTE ACCENT).

Even though they represent the same thing to a human, they don't compare
as equal. But if you normalize them to the same form, they will.

For more information, look at the unicodedata module's documentation:

-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Process "Killed"

[Matt Nordhoff]

dieter wrote:
> Hi,
> 
> Overview
> ===
> 
> I'm doing some simple file manipulation work and the process gets
> "Killed" everytime I run it. No traceback, no segfault... just the
> word "Killed" in the bash shell and the process ends. The first few
> batch runs would only succeed with one or two files being processed
> (out of 60) before the process was "Killed". Now it makes no
> successful progress at all. Just a little processing then "Killed".

That isn't a Python thing. Run "sleep 60" in one shell, then "kill -9"
the process in another shell, and you'll get the same message.

I know my shared web host has a daemon that does that to processes that
consume too many resources.

Wait a minute. If you ran this multiple times, won't it have removed the
first two lines from the first files multiple times, deleting some data
you actually care about? I hope you have backups...

> Question
> ===
> 
> Any Ideas? Is there a buffer limitation? Do you think it could be the
> filesystem?
> Any suggestions appreciated Thanks.
> 
> 
> The code I'm running:
> ==
> 
> from glob import glob
> 
> def manipFiles():
> filePathList = glob('/data/ascii/*.dat')

If that dir is very large, that could be slow. Both because glob will
run a regexp over every filename, and because it will return a list of
every file that matches.

If you have Python 2.5, you could use glob.iglob() instead of
glob.glob(), which returns an iterator instead of a list.

> for filePath in filePathList:
> f = open(filePath, 'r')
> lines = f.readlines()[2:]

This reads the entire file into memory. Even better, I bet slicing
copies the list object temporarily, before the first one is destroyed.

> f.close()
> f = open(filePath, 'w')
> f.writelines(lines)
> f.close()
> print file

This is unrelated, but "print file" will just say "",
because it's the name of a built-in object, and you didn't assign to it
(which you shouldn't anyway).


Actually, if you *only* ran that exact code, it should exit almost
instantly, since it does one import, defines a function, but doesn't
actually call anything. ;-)

> Sample lines in File:
> 
> 
> # time, ap, bp, as, bs, price, vol, size, seq, isUpLast, isUpVol,
> isCancel
> 
> 1062993789 0 0 0 0 1022.75 1 1 0 1 0 0
> 1073883668 1120 1119.75 28 33 0 0 0 0 0 0 0
> 
> 
> Other Info
> 
> 
> - The file sizes range from 76 Kb to 146 Mb
> - I'm running on a Gentoo Linux OS
> - The filesystem is partitioned and using: XFS for the data
> repository, Reiser3 for all else.

How about this version? (note: untested)

import glob
import os

def manipFiles():
# If you don't have Python 2.5, use "glob.glob" instead.
filePaths = glob.iglob('/data/ascii/*.dat')
for filePath in filePaths:
print filePath
fin = open(filePath, 'rb')
fout = open(filePath + '.out', 'wb')
# Discard two lines
fin.next(); fin.next()
fout.writelines(fin)
fin.close()
fout.close()
os.rename(filePath + '.out', filePath)

I don't know how light it will be on CPU, but it should use very little
memory (unless you have some extremely long lines, I guess). You could
write a version that just used .read() and .write() in chunks

Also, it temporarily duplicates "whatever.dat" to "whatever.dat.out",
and if "whatever.dat.out" already exists, it will blindly overwrite it.

Also, if this is anything but a one-shot script, you should use
"try...finally" statements to make sure the file objects get closed (or,
in Python 2.5, the "with" statement).
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python multimap

[Matt Nordhoff]

brad wrote:
> Recently had a need to us a multimap container in C++. I now need to
> write equivalent Python code. How does Python handle this?
> 
> k['1'] = 'Tom'
> k['1'] = 'Bob'
> k['1'] = 'Joe'
> ...
> 
> Same key, but different values. No overwrites either They all must
> be inserted into the container
> 
> Thanks,
> Brad

I don't know if this is exactly equivalent, but what about using a
defaultdict like this?

>>> from collections import defaultdict
>>> k = defaultdict(list)
>>> k['1'].append('Tom')
>>> k['1'].append('Bob')
>>> k['1'].append('Joe')
>>> k['1']
['Tom', 'Bob', 'Joe']
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleJson is slow .... is there any C Compiled version ?

[Matt Nordhoff]

sanket wrote:
> Hello All,
> 
> I have created an API which fetches some data from the database.
> I am using simplejson to encode it and return it back.
> 
> Now the problem is that, this API is being called for millions of
> times in a sequence.
> I ran a profiler and saw that most of the time is consumed in encoding
> my database results in to json.
> So I was just wondering is there any C compiled version of simplejson
> is available?
> or any help regarding this issue would be appreciated.
> 
> Thank you,
> Sanket

simplejson is not the only JSON library out there. For example, there's
python-cjson, which is written entirely in C:



There's also an enhanced version of it:



I think simplejson has some small, optional C bits that will improve
performance if you compile them.

Also, be aware that I think simplejson is being integrated into the
stdlib in Python 2.6.

Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: redirecting output of process to a file using subprocess.Popen()

[Matt Nordhoff]

skeept wrote:
> On Jul 9, 7:32 pm, [EMAIL PROTECTED] wrote:
>> I am trying to redirect stderr of a process to a temporary file and
>> then read back the contents of the file, all in the same python
>> script. As a simple exercise, I launched /bin/ls but this doesn't
>> work:
>>
>> #!/usr/bin/python
>> import subprocess as proc
>> import tempfile
>> name = tempfile.NamedTemporaryFile(mode='w+b')
>> print 'name is '+ name.name
>>
>> cmd = []
>> cmd.append('/bin/ls')
>> cmd.append('-l')
>> cmd.append('/tmp')
>> p = proc.Popen(cmd, stdout=name, stderr=proc.STDOUT, close_fds=True)
>> while True:
>>ret = p.poll()
>>if (ret is not None):
>>   output = name.readlines()
>>   print 'out = ', output
>>   break
>>
>> $python sub.py
>> name is /tmp/tmpjz4NJY
>> out =  []
>>
>> I tried calling flush() on the file object but this didn't help
>> either. Tried closing and re-opening the file, but closing the file
>> object results in it getting deleted. Can the above be made to work by
>> using tempfiles?
>>
>> thanks
> 
> 
> your script works just fine.
> The problem is that you have to move to the beggining of the file to
> read the actual contents of
> the file.
> name.seek(0)
> 
> The whole program would be:
> 
> #!/usr/bin/python
> import subprocess as proc
> import tempfile
> name = tempfile.NamedTemporaryFile(mode='w+b')
> print 'name is '+ name.name
> 
> cmd = []
> cmd.append('/bin/ls')
> cmd.append('-l')
> cmd.append('/tmp')
> p = proc.Popen(cmd, stdout=name, stderr=proc.STDOUT, close_fds=True)
> while True:
>ret = p.poll()
>if (ret is not None):
>   name.seek(0)
>   output = name.readlines()
>   print 'out = ', output
>   break

This is an aside, but why the loop?

...
p = ...
p.wait()
name.seek(0)
...

(p.wait() returns p.returncode, just like p.poll() does, but you aren't
using it anyway...)
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Confused yet again: Very Newbie Question

[Matt Nordhoff]

Jerry Hill wrote:
> On Mon, Jul 7, 2008 at 7:30 AM, mcl <[EMAIL PROTECTED]> wrote:
>> I did not think you had to make the distinction between 'byvar' and
>> 'byref' as in Basic.
> 
> Python does not use "call by value" or "call by reference" semantics.
> Instead, python's model is "call by object".  See this writeup for
> some details: http://effbot.org/zone/call-by-object.htm

Ah-ha! That's the article I've been trying to find! I can never remember
"effbot"'s name. Thanks!
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: I am looking for svn library(module)

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> Hi,
> 
> I am looking fo svn library(module) which is used in the svn-
> mailer(http://opensource.perlig.de/svnmailer/) project. Does anybody
> know where can I find it(download url)? This is information which I
> received from python error:
> 
> from svn import core as svn_core
> ImportError: No module named svn
> 
> Best regards

Those would be the bindings that come with Subversion. They're covered
at the end of ./INSTALL in the source tree (which just points you to
./subversion/bindings/swig/INSTALL). If you installed svn with a package
manager, there's probably a package for them (e.g. python-subversion on
Ubuntu).
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Confused yet again: Very Newbie Question

[Matt Nordhoff]

mcl wrote:
> On 7 Jul, 13:09, Jeff <[EMAIL PROTECTED]> wrote:
>> When you call c3.createJoe(c1.fred), you are passing a copy of the
>> value stored in c1.fred to your function.  Python passes function
>> parameters by value.  The function will not destructively modify its
>> arguments; you must expliticly state your intention to modify an
>> object:
>>
>> class one():
>> fred = 'fred'
>>
>> class three():
>> def createJoe(self, myName):
>> return "Joe"
>>
>> def main():
>> c1 = one()
>> c3 = three()
>> c1.fred = c3.createJoe() # Modify c1's attribute, fred, with the
>> return value from c3.createJoe
> 
> Thank you very much for your helpful replies.
> 
> Two questions:
> One:
> My use of classes is because I want two classes one for  global
> variables and one for global functions.

That's odd. Usually, you shouldn't be using any globals at all. Callers
should use their own local variables, or instances or whatever.

> A function may set multiple global variable values, so what is the
> best way to do this as 'return' only appears to be able to set one
> value.

You can return a tuple, and then use tuple unpacking.

>>> def f():
... return 1, 2
>>> one, two = f()
>>> one
1
>>> two
2

> Two:
> I am sorry but I do not understand the significance defining a Class
> as:
> >>> class MyClass(object):
>   what is object ?
> 
> I am using python with Google App Engine, and I only have Alex
> Martelli's book up to Python 2.2, if they has any relevance ?

By inheriting from object, your class becomes a new-style class.
Otherwise, it's old-style. Honestly, I don't know all of the
differences, but new-style classes are better. For one thing, old-style
classes don't support properties or super().

They were introduced in Python 2.2. Your Python 2.2 book probably
centers around old-style classes.

> Thanks again for your help. I now understand what my mistakes were and
> why I was not getting the results I had expected.
> 
> Richard
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Confused yet again: Very Newbie Question

[Matt Nordhoff]

mcl wrote:
> Why can I not the change the value of a variable in another class,
> when I have passed it via a parameter list.
> 
> I am sure I am being stupid, but I thought passed objects were Read/
> Write

In Python, there are names which are bound to objects. Doing "foo = bar"
and then "foo = spam" (re)binds the name "foo" to the same object as
"spam" is bound to. This doesn't have any effect on any other names that
were bound to the same object as "bar".

> eg
> 
> #!/usr/bin/python
> 
> class one():  #my Global Vars
> fred = 'fred'
> 
> class three():
> def createJoe(self, myName):

Here, the local name "myName" is bound to the same string object as
one.fred.

> c1 = one()
> myName = 'Joe'  #* Question why does this
> not change variable fred in 'class one'

Here, you rebind the local name "myName" to the string object 'Joe'.
This doesn't change what one.fred is bound to.

> print 'Three(Local): ' + myName + 'Three(Global): ' +
> c1.fred
> 
> def main():
> c1 = one()
> c3 =three()
> c3.createJoe(c1.fred)
> 
> 
> if __name__ == '__main__' : main()
> 
> Results:
> Three(Local): JoeThree(Global): fred
> 
> 'fred' in 'class one' does not get changed to 'joe' in 'class three'
> 'createJoe', even though I have passed 'class one' 'fred' to
> 'createJoe'
> 
> I hope this makes sense.
> 
> I did not think you had to make the distinction between 'byvar' and
> 'byref' as in Basic.
> 
> Thanks
> 
> Richard

There are a couple good articles about this online. The one below is
lengthy and has ASCII art, but I don't remember if I liked it.



BTW, in this example, there doesn't seem to be any need for you to be
using classes. Also, you should always use new-style classes unless you
have a specific reason not to

>>> class MyClass(object):
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: mirroring files and data via http

[Matt Nordhoff]

Steve Potter wrote:
> On Jul 6, 8:19 pm, Matt Nordhoff <[EMAIL PROTECTED]> wrote:
>> Steve Potter wrote:
>>> I'm working on a project to create a central administration interface
>>> for several websites located on different physical servers.
>>> You can think of the websites as a blog type application. My
>>> administration application will be used to create new blog posts with
>>> associated media (images, etc..)
>>> So I am thinking to setting up a script on each of the sites to "phone
>>> home" once per day and download any new posts and media files.
>>> I am thinking of transmitting the post data in an xml format that
>>> could then be decoded an recorded into the database on the slave
>>> site.   Are there any easy ways to encode a python dictionary to and
>>> from xml?
>>> For the media files I am thinking that the administration interface
>>> would also provide an xml file with a list of all of the media files
>>> required along with an http path to retrieve them and a checksum of
>>> some sort to verify that they were downloaded correctly.
>>> Basically I am trying to figure out if anything already exists to
>>> perform some of these functions or if I have to make them from
>>> scratch.  I know none of it should be too hard, but I hate to re-
>>> invent the wheel.
>>> Thanks,
>>> Steven Potter
>> It sounds like JSON would be perfect for this. For working with it in
>> Python, the simplejson module is popular:
>>
>> <http://pypi.python.org/pypi/simplejson>
>> --
> 
> Matt,
> 
> You are correct JSON would be much easier to deal with than xml.  That
> takes care of several of the problems.
> 
> Now it really just comes down to the file transfer from one server to
> the other and a verification of some sort that the file transfer was
> not corrupt.  Is there a python module that takes care of file
> downloads and verification?
> 
> Thanks,
> 
> Steve

Not automatically, but this should be easy to do yourself. You could
generate a simple JSON or plain text file listing URLs and hashes, and
then use urllib or urllib2 to download them, and md5, sha1 or (in Python
2.5) hashlib to verify them. All of those modules are in the standard
library.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: mirroring files and data via http

[Matt Nordhoff]

Steve Potter wrote:
> I'm working on a project to create a central administration interface
> for several websites located on different physical servers.
> 
> You can think of the websites as a blog type application. My
> administration application will be used to create new blog posts with
> associated media (images, etc..)
> 
> So I am thinking to setting up a script on each of the sites to "phone
> home" once per day and download any new posts and media files.
> 
> I am thinking of transmitting the post data in an xml format that
> could then be decoded an recorded into the database on the slave
> site.   Are there any easy ways to encode a python dictionary to and
> from xml?
> 
> For the media files I am thinking that the administration interface
> would also provide an xml file with a list of all of the media files
> required along with an http path to retrieve them and a checksum of
> some sort to verify that they were downloaded correctly.
> 
> Basically I am trying to figure out if anything already exists to
> perform some of these functions or if I have to make them from
> scratch.  I know none of it should be too hard, but I hate to re-
> invent the wheel.
> 
> Thanks,
> 
> Steven Potter

It sounds like JSON would be perfect for this. For working with it in
Python, the simplejson module is popular:


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Are the following supported in scipy.sparse

[Matt Nordhoff]

dingo_1980 wrote:
> I wanted to know if scipy.sparse support or will support the following
> functions which are available in scipy.linalg or scipy or numpy:
> 
> Inverse
> Cholesky
> SVD
> multiply
> power
> append
> eig
> concatenate
> 
> Thanks...

You should probably ask on a SciPy mailing list.


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using just the Mako part of Pylons?

[Matt Nordhoff]

John Salerno wrote:
> Bruno Desthuilliers wrote:
>> John Salerno a écrit :
>>> I just installed Pylons onto my hosting server so I could try out
>>> templating with Mako, but it seems a little more complicated than that.
>>
>> Err... Actually, it's certainly a little less complicated than that.
>> First point: Mako is totally independant from Pylons. Second point:
>> you don't need any web server to use Mako - it's just another Python
>> package.
> 
> Well, so far I have installed Pylons onto my web server, but a test Mako
> template isn't rendering, it just prints the text to the browser.
> There's got to be something more to do to get it all hooked up so that
> it works.

What do you mean by "the text"? The source code of the template? A
random insult? The generated page? If it's the last, you forgot to set a
Content-Type header.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Windows OS , Bizarre File Pointer Fact

[Matt Nordhoff]

Taygun Kekec wrote:
> Code :
> #!/usr/bin/python
> # -*- coding: utf-8 -*-
> import os
> 
> if os.name == 'nt':
> OS_Selection = 0
> elif os.name == 'posix':
> OS_Selection = 1
> else :
> OS_Selection = 1
> 
> del_cmd_os = ( "del","rm")
> filelist = ("ddd.txt","eee.txt","fff.txt")
> 
> # Creating Files
> for elem in filelist:
> open( elem, 'w').write("Selam")
> 
> #
> # Removing Files
> for elem in filelist:
> fp = open( elem, 'r')
> os.system( "%s %s" % (del_cmd_os[OS_Selection], fp.name))
> fp.close()
> #
> 
> Run this code on Windows XP , it says "the file is being used by
> another process" and fails to delete the file. I tried replacing :
> #
> for elem in filelist:
> open( elem, 'w').write("Selam")
> #
> with :
> #
> for elem in filelist:
> fp = open( elem, 'w')
> fp.write("Selam")
> fp.close()
> #
> 
> in case of any interpreter file pointer destructor failure but it
> didnt change anything.
> Do you have any idea why my files cannot be deleted from my disk with
> 2nd part of my code ?

Why are you executing another program just to delete a file?

>>> import os
>>> os.remove('some/file.txt')
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird local variables behaviors

[Matt Nordhoff]

Sebastjan Trepca wrote:
> Hey,
> 
> can someone please explain this behavior:
> 
> The code:
> 
> def test1(value=1):
> def inner():
> print value
> inner()
> 
> 
> def test2(value=2):
> def inner():
> value = value
> inner()
> 
> test1()
> test2()
> 
> [EMAIL PROTECTED] ~/dev/tests]$ python locals.py
> 1
> Traceback (most recent call last):
>   File "locals.py", line 13, in 
> test2()
>   File "locals.py", line 10, in test2
> inner()
>   File "locals.py", line 9, in inner
> value = value
> UnboundLocalError: local variable 'value' referenced before assignment
> 
> Why can't he find the variable in the second case?
> 
> 
> Thanks, Sebastjan

Python doesn't like when you read a variable that exists in an outer
scope, then try to assign to it in this scope.

(When you do "a = b", "b" is processed first. In this case, Python
doesn't find a "value" variable in this scope, so it checks the outer
scope, and does find it. But then when it gets to the "a = " part...
well, I don't know, but it doesn't like it.)

In Python 3 (and maybe 2.6), you'll be able to put "nonlocal value" in
inner() to tell it to use the outer scope. (It's similar to the "global"
keyword, but uses the next outer scope as opposed to the global scope.)
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: images on the web

[Matt Nordhoff]

Matt Nordhoff wrote:
> chris wrote:
>> I'm creating a data plot and need to display the image to a web page.
>> What's the best way of doing this without having to save the image to
>> disk? I already have a mod_python script that outputs the data in
>> tabular format, but I haven't been able to find anything on adding a
>> generated image.
> 
> You could use data: URIs [1].
> 
> For example, a 43-byte single pixel GIF becomes this URI:
> 
> 
> 
> They don't have universal browser support, but that might not be a
> problem in this case.
> 
> As for generating them with Python, I'm not sure... I just used Hixie's
> data: URI kitchen [2] for the above example.
> 
> [1] <http://tools.ietf.org/html/rfc2397>
> [2] <http://software.hixie.ch/utilities/cgi/data/data>

Oh.. As <http://bitworking.org/news/Sparklines_in_data_URIs_in_Python>
shows, the reason I couldn't find a data: URI Python library is because
they're utterly trivial to generate:

import base64
import urllib

raw_data = create_gif()
uri = 'data:image/gif;base64,' + urllib.quote(base64.b64encode(raw_data))

(And it's even simpler if you leave out the base64-encoding.)
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: images on the web

[Matt Nordhoff]

chris wrote:
> I'm creating a data plot and need to display the image to a web page.
> What's the best way of doing this without having to save the image to
> disk? I already have a mod_python script that outputs the data in
> tabular format, but I haven't been able to find anything on adding a
> generated image.

You could use data: URIs [1].

For example, a 43-byte single pixel GIF becomes this URI:



They don't have universal browser support, but that might not be a
problem in this case.

As for generating them with Python, I'm not sure... I just used Hixie's
data: URI kitchen [2] for the above example.

[1] 
[2] 
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Simple Class/Variable passing question

[Matt Nordhoff]

monkeyboy wrote:
> Hello,
> 
> I'm new to python, and PythonCard. In the code below, I'm trying to
> create a member variable (self.currValue) of the class, then just pass
> it to a simple function (MainOutputRoutine) to increment it. I thought
> Python "passed by reference" all variables, but the member variable
> doesn't seem to be incremented. If I make the function just increment
> the member variable directly, without passing it in, it works fine?
> 
> In the code below, "self.currValue" stays at 5, and value evaluates to
> 1? Any insight would be appreciated...
> 
> 
> class TestModel(model.Background):
> 
> def on_initialize(self,event):
> self.currValue = 5
> 
> def on_startBtn_mouseClick(self, event):
> self.MainOutputRoutine(self.currValue)
> self.OutputRoutine(self.currValue)
> 
> def OutputRoutine(self,value):
> self.components.txtBox1.text = str(value)
> 
> def MainOutputRoutine(self,value):
> value = value + 1

That's not how Python works. When you call
"self.MainOutputRoutine(self.currValue)", in that method's scope, the
local name "value" points to the same object as self.currValue does.
When you do "value = value + 1", the local name "value" now points to a
different object. That has no bearing on self.currValue.

Err, I can't find a guide here. Um, read the language spec? I dunno.

However:

>>> my_list = [1]
>>> def function(l):
... l.append(2)
>>> function(my_list)
>>> my_list
[1, 2]

That's because function() is *mutating* the list; it's not changing what
the "l" name points to. It's calling the "append" method of the list
object, which changes said list object. If it were doing, say, "l = 42",
that would only rebind the function's local name "l":

>>> my_list = [1]
>>> def function(l):
... l = 42
>>> function(my_list)
>>> my_list
[1]

Note that strings and integers are immutable, so whenever you think
you're mutating them (e.g. "s.replace('a', 'b')" or "i += 1"), you're
actually getting a whole new, different object, with all that that implies.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: python/ruby question..

[Matt Nordhoff]

Mensanator wrote:
> On Jun 18, 10:33�pm, "bruce" <[EMAIL PROTECTED]> wrote:
>> hi...
>>
>> can someone point me to where/how i would go about calling a ruby app from a
>> python app, and having the python app being able to get a returned value
>> from the ruby script.
>>
>> something like
>>
>> test.py
>> �a = os.exec(testruby.rb)
>>
>> testruby.py
>> �foo = 9
>> �return foo
>>
>> i know this doesn't work... but i've been searching for hours on this with
>> no luck (and yeah, i'm relatively new to both ruby/python!!)
>>
>> thanks
> 
> Well, I don't know anything about Ruby, but here's
> how I do it for C programs (compiled to .exe that
> write to stdout).
> 
> 
> import os
> factor_program = 'factor! -d200 ' # factor!.exe from MIRACL
> 
> n =
> '50818429800343305993022114330311033271249313957919046352679206262204589342623811236647989889145173098650749'
> 
> # call external program and capture stdout
> the_output = os.popen(factor_program+n).readlines()
> 
> print 'n: %s' % n
> for i in the_output:
> print i,



You're supposed to use the subprocess module.

In this case, something like:

import subprocess
factor_program = ['factor!', '-d200']

...

p = subprocess.Popen(factor_program + [n], stdout=subprocess.PIPE)
p.wait() # wait for it to finish; not sure how necessary it is
the_output = p.stdout.readlines()

See subprocess's documentation [1], which includes guides on replacing
os.popen* and other functions with it.

[1] 
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: reading from an a gzip file

[Matt Nordhoff]

Nader wrote:
> Hello,
> 
> I have a gzip file and I try to read from this file withe the next
> statements:
> 
>  gunziped_file = gzip.GzipFile('gzip-file')
>  input_file = open(gunziped_file,'r')
> 
> But I get the nezt error message:
> 
> Traceback (most recent call last):
>   File "read_sfloc_files.py", line 131, in ?
> input_file = open(gunziped_file,'r')
> TypeError: coercing to Unicode: need string or buffer, instance found
> 
> I think that I do some mistake. Would some body tell me what is my
> mistake?
> 
> Nader

gzip.GzipFile() creates a file-like object. You don't call open() on it;
you use it directly.

$ echo foo >test
$ gzip test
$ python
>>> import gzip
>>> gfh = gzip.GzipFile('test.gz', 'rb')
>>> gfh.read()
'foo\n'
>>> gfh.close()
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: write Python dict (mb with unicode) to a file

[Matt Nordhoff]

dmitrey wrote:
> hi all,
> what's the best way to write Python dictionary to a file?
> 
> (and then read)
> 
> There could be unicode field names and values encountered.
> Thank you in advance, D.

pickle/cPickle, perhaps, if you're willing to trust the file (since it's
basically eval()ed)? Or JSON (use simplejson or the enhanced version of
cjson), though I doubt it would be super-fast.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3000 vs. Python 2.x

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> As a new comer to Python I was wondering which is the best to start
> learning.  I've read that a number of significant features have
> changed between the two versions.  Yet, the majority of Python
> programs out in the world are 2.x and it would be nice to understand
> those as well.  Thanks for all the help.
> 
> Creosote,

You should learn Python 2. Python 3 is only in alpha/beta, and it won't
be very relevant for several years.

Python 3 isn't a whole new language; it just breaks backwards
compatibility to clean up old warts and make other improvements. It
won't be too hard to transition to it when you decide to.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does this path exist?

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> I wanted to ask for ways to test whether a path exists. I usually use
> os.path.exists(), which does a stat call on the path and returns True
> if it succeeds, or False if it fails (catches os.error). But stat
> calls don't fail only when a path doesn't exist. I see that, at least
> on Windows, the instance of the exception has an attribute 'errno' set
> to 2 when it fails because the path doesn't exist. Is it a portable
> solution to rely on this (haven't tried it on Linux)? Are there other
> ways of testing whether a path exists?
> 
> Thanks,
> Sebastian

"import errno" and see if the exception's errno attribute is set to
errno.ENOENT (which is, yes, 2). It is portable.

If you Google [ENOENT Windows] or whatever, there are some differences
on different platforms, but not many.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: module import problem

[Matt Nordhoff]

Milos Prudek wrote:
> I have a Kubuntu upgrade script that fails to run:
> 
> File "/tmp/kde-root//DistUpgradeFetcherCore.py", 
> line 34, in 
> import GnuPGInterface
> ImportError
> No module named GnuPGInterface
> 
> I got a folder /usr/share/python-support/python-gnupginterface with 
> a "GnuPGInterface.py" but no __init__.py.
> 
> In python command line, print sys.path shows that /usr/share/python-support/ 
> is not among python paths.
> 
> If I cd into /usr/share/python-support/python-gnupginterface and launch 
> Python 
> I can "import GnuPGInterface". But when I run DistUpgradeFetcherCore.py in 
> that folder it always fails with No module named GnuPGInterface.
> 
>  I do not know much about setting python path. 

This doesn't help solve your problem, but /usr/share/python-support
doesn't need to be in Python's path because python-support puts symlinks
to it in /usr/share/python*/site-packages.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Get all the instances of one class

[Matt Nordhoff]

Tommy Nordgren wrote:
> class MyClass : a_base_class
>   memberlist=[]
> 
> #  Insert object in memberlist when created;
> #  note: objects won't be garbage collected until removed from memberlist.

Just to say, if you wanted to go about it that way, you could avoid the
garbage collection problem by using weakrefs:


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: python without while and other "explosive" statements

[Matt Nordhoff]

ivo talvet wrote:
> Hello,
> 
> Is it possible to have a python which not handle the execution of
> "while", "for", and other loop statements ? I would like to allow
> remote execution of python on a public irc channel, so i'm looking for
> techniques which would do so people won't be able to crash my computer
> (while 1: os.fork(1)), or at least won't won't freeze my python in a
> infinite loop, make it unresponsive. Is there a compiling option (or
> better, something i can get with apt-get cos i think compiling myself
> and handle all the metastuff myself is somehow dirty) for have a
> "secure python" (you can guess what i mean by "secure" in my case) or
> must i touch myself the source disable some code lines ? If last
> solution, which modifications in which files should i do ? (sorry for
> my bad english)
> 
> Thanks.
> 
> Ivo

 is a pastebin-like website that lets people upload
code in a dozen different languages, and it runs it. They have info up
about how it's secure at .

I'm pretty sure there's another similar website that's specific to
Python, but I can't remember it. Anyway, it was similar, using virtual
machines, a firewall, and killing stuff that ran for longer than 20
seconds, IIRC.

(Thanks for the link to codepad, habnabit.)
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is using range() in for loops really Pythonic?

[Matt Nordhoff]

John Salerno wrote:
> I know it's popular and very handy, but I'm curious if there are purists
> out there who think that using something like:
> 
> for x in range(10):
> #do something 10 times
> 
> is unPythonic. The reason I ask is because the structure of the for loop
> seems to be for iterating through a sequence. It seems somewhat
> artificial to use the for loop to do something a certain number of
> times, like above.
> 
> Anyone out there refuse to use it this way, or is it just impossible to
> avoid?

Well, you should use "xrange(10)" instead of "range(10)". While range()
returns a list of every single number, xrange() returns an iterable
object that only generates them on-demand, so xrange(1) and
xrange(sys.maxint) will use the same amount of RAM.

(Not that it matters when it's only 10 items, of course.)

Other than that, "for i in xrange(10)" is a standard Python idiom, even
though it is a bit weird.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python, are you ill?

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> You
> can't get out of the code block with pressing the Enter key; you have
> to press Ctrl+Z (if you're in Linux) in order to get out of that code
> block, which then throws you back to the Linux command line, but
> before that it prints this line
> 
> [1]+  Stopped python

You can use Ctrl+C and Python will stop what it's doing and go back to
its main prompt.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Simple question

[Matt Nordhoff]

Gandalf wrote:
> my server is my computer and all i did way to install python on it.

But what web server program are you using? Apache? IIS? Lighttpd?
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something with Python not having interfaces?

[Matt Nordhoff]

Mike Driscoll wrote:
> On May 6, 8:44 am, jmDesktop <[EMAIL PROTECTED]> wrote:
>> Studying OOP and noticed that Python does not have Interfaces.  Is
>> that correct?  Is my schooling for nought on these OOP concepts if I
>> use Python.  Am I losing something if I don't use the "typical" oop
>> constructs found in other languages (Java, C# come to mind.)  I'm
>> afraid that if I never use them I'll lose them and when I need them
>> for something beside Python, I'll be lost.  Thank you.
> 
> In my school, we didn't even discuss the concept of interfaces (except
> for CLI and GUI, that is). So I looked it up. I assume you are
> referring to something like what's found here:
> 
> http://java.sun.com/docs/books/tutorial/java/concepts/interface.html
> 
> If so, then it looks like an Interface is a generic class with method
> stubs. You can do that with Python just as easily by creating empty
> methods with just the "pass" keyword. It also reminds me of
> Decorators...so you might want to look at those.

Instead of just using "pass", you should "raise NotImplementedError".

>From the docs: "In user defined base classes, abstract methods should
raise this exception when they require derived classes to override the
method."

> Since it's just a construct to implement polymorphism, I don't think
> you'll lose anything. However, Python does not require you to re-
> implement every method of the class it is inheriting from. You can
> just override those that you want and leave the others alone.
> 
> Hopefully I understand this correctly...otherwise, just ignore my
> babbling and hand waving.
> 
> Mike
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: removing extension

[Matt Nordhoff]

Arnaud Delobelle wrote:
> More simply, use the rsplit() method of strings:
> 
 path = r'C:\myimages\imageone.jpg'
 path.rsplit('.', 1)
> ['C:\\myimages\\imageone', 'jpg']
> 
> 
 path = r"C:\blahblah.blah\images.20.jpg"
 path.rsplit('.', 1)
> ['C:\\blahblah.blah\\images.20', 'jpg']
> 
> HTH

There's os.path.splitext(), which probably does just about exactly that.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is 2006 too old for a book on Python?

[Matt Nordhoff]

jmDesktop wrote:
> Hi, I wanted to buy a book on Python, but am concerned that some of
> them are too old.  One I had come to after much research was Core
> Python by Wesley Chun.  I looked at many others, but actually saw this
> one in the store and liked it.  However, it is from 2006.  I know
> there is free documentation and Dive Into Python.  I just liked the
> one in question and was hoping for thoughts on the age of it.  I am
> using Python 2.5.x and don' t know how different 2.3, 2,4 is from it.
> 
> Thank you.

Python 2.x releases maintain backwards compatibility. There are nice
features that have come since 2.3, but it's not like it's a different
language. You could learn from a slightly old book, then do some
research to find out what you missed. ('with' statement, ternary
operator, smaller things in the stdlib. Maybe generators? What else?)

Now, Python 3 will break backwards compatibility and make more
significant changes than usual, but it will still be basically the same
language, so the same thing applies. And anyway, it won't be very
relevant for a few years.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: problem with unicode

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> Hi everybody,
> 
> I'm using the win32 console and have the following short program
> excerpt
> 
> # media is a binary string (mysql escaped zipped file)
> 
>>> print media
> xワユロ[ヨ...
> (works)
> 
>>> print unicode(media)
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position
> 1: ordinal not in range(128)
> (ok i guess print assumes you want to print to ascii)



> I don't know what to do. I just want to concatenate two string where
> apparently one is a binary string, the other one is a unicode string
> and I always seem to get this error.
> 
> Any help is appreciated :)

You should read the Python Unicode documentation, such as:





and maybe a non-Python-specific article:


-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: library to do easy shell scripting in Python

[Matt Nordhoff]

Wow, this message turned out to be *LONG*. And it also took a long time
to write. But I had fun with it, so ok. :-)

Michael Torrie wrote:
> Recently a post that mentioned a recipe that extended subprocess to
> allow killable processes caused me to do some thinking.  Some of my
> larger bash scripts are starting to become a bit unwieldy (hundreds of
> lines of code).  Yet for many things bash just works out so well because
> it is so close to the file system and processes.  As part of another
> project, I now have need of a really good library to make it almost as
> easy to do things in Python as it is in Bash.  With a simple wrapper
> around subprocess, I'm pretty much able to do most things.  Most of my
> complicated bash hackery involves using awk, sed, grep, and cut to
> process text, which python does quite nicely, thank you very much.  But
> there's a few things to add.
> 
> To wit, I'm wanting to write a library that can deal with the following
> things:
> 
>   - spawn a process, feed it std in, get stdout, stderr, and err code.
> This is largely already accomplished by subprocess

It is accomplished by subprocess.Popen:

The 'communicate' method handles stdin, stdout and stderr, waiting for
the process to terminate.

The 'wait' method just waits for the process to terminate and returns
the return code.

The 'returncode' attribute contains the return code (or None if the
process hasn't terminated yet).

You could write a convenience wrapper function if you want to do this in
a more terse way.

>   - spawn off processes as background daemons

Couldn't you do this with subprocess by doing subprocess.Popen([prog])
and, well, nothing else? (You may have/want to set stdin/stdout/stderr
too. I dunno.)

>   - spawn multiple processes and pipe output to input.
> - can do fancier things like bash does, like combine stderr/stdout,
>   switch stderr/stdout, redirects to and from files

That's possible with subprocess.

See this paragraph of :

> stdin, stdout and stderr specify the executed programs' standard input, 
> standard output and standard error file handles, respectively. Valid values 
> are PIPE, an existing file descriptor (a positive integer), an existing file 
> object, and None. PIPE indicates that a new pipe to the child should be 
> created. With None, no redirection will occur; the child's file handles will 
> be inherited from the parent. Additionally, stderr can be STDOUT, which 
> indicates that the stderr data from the applications should be captured into 
> the same file handle as for stdout.

And also . Not the least
verbose, but pretty simple, and I bet it can do anything bash can.

> - transparently allow a python function or object to be a part of
>   the pipeline at any stage.

Hmmm. I can't think very well at the moment, but you could create
file-like objects that do...I dunno, callbacks or something.

Simple and incomplete mockup:

class Pipe(object):
def __init__(self, from_fh, to_fh, from_callback=None,
to_callback=None):
self.from_fh = from_fh
self.to_fh = to_fh
self.from_callback = from_callback
self.to_callback = to_callback

def read(self, *args, **kwargs):
data = self.from_fh.read(*args, **kwargs)
if self.from_callback is not None:
self.from_callback(data)
return data

def write(self, data):
# XXX Call the callback before or after the data is actually
written?
if self.to_callback is not None:
self.to_callback(data)
return self.to_fh.write(data)

That just passes input and output through itself, also passing it to
callback functions. You'd have to add all the other methods too, like
readline and __iter__... Maybe inheriting from 'file' would get most of
them. I dunno how it works internally.

> Questions include, how would one design the interface for things, like
> assembling pipes?  Several ideas include:
> 
> pipe([prog1,args],[prog2,args],...)
> 
> or
> 
> run([prog1,args]).pipe([prog2,args]).pipe(...)
> 
> The former doesn't deal very well with re-plumbing of the pipes, nor is
> there an easy way to redirect to and from a file. The second syntax is
> more flexible but a bit cumbersome.  Also it doesn't allow redirection
> or flexible plumbing either.
> 
> Any ideas on how I could design this?

Ok, the below is an edited-down, more formal-sounding brain dump.

Idea 1:

>>> run([prog, args], from_fh, to_fh, from_callback, to_callback).run(...)

It would basically just automate the construction of the intermediary
pipe objects suggested above.

It could also be done with tuples, like:

>>> run([prog, args], (from_fh, to_fh), (from_callback,
to_callback)).run(...)

Idea 2:

This one would parse a list similar to a bash command line.

run('prog', '>>out', '|', 'other_prog', 'arg', '>', 'foo.txt')

Which would be like a bash:

`prog 2>&1 | other_prog arg >foo.txt

Re: Lists: why is this behavior different for index and slice assignments?

[Matt Nordhoff]

John Salerno wrote:
>> replaces the elements in the slice by assigned elements.
> 
> 
> I don't understand the second part of that sentence. I'm assuming "it"
> refers to the list being assigned, "replaces the elements" is
> self-evident, but what does "by assigned elements" refer to? It seems
> when you assign a list to a list slice, nothing gets replaced, the slice
> just gets deleted.

>>> x = range(5)
>>> x[0:3] = ["a", "b"]
>>> x
['a', 'b', 3, 4]

Here, '= ["a", "b"]' replaces x[0:3] with ["a", "b"]. When you do '=
[]', it replaces them with nothing.
-- 
--
http://mail.python.org/mailman/listinfo/python-list

Re: import statement convention

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> By convention, I've read, your module begins with its import
> statements. Is this always sensible?
> 
> I put imports that are needed for testing in the test code at the end
> of the module. If only a bit of the module has a visual interface, why
> pollute the global namespace with 'from Tkinter import *'? Wouldn't
> that be better done in a separate class or function?
> 
> Can we do a better job with a thoughtful rewrite of this convention?

I don't think anyone has been beheaded for breaking from convention when
there's a good reason... The Zen of Python does say "Although
practicality beats purity.".

(One useful related thing is the lazy import modules developed by e.g.
Bazaar and Mercurial, where you can "import foo" at the top level, but
the import won't really be done until it's actually used.)
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Destructor?

[Matt Nordhoff]

Matt Nordhoff wrote:
> Gabriel Rossetti wrote:
>> Hello everyone,
>>
>> we are writing an application that needs some cleanup to be done if the 
>> application is quit, normally (normal termination) or by a signal like 
>> SIGINT or SIGTERM. I know that the __del__ method exists, but unless I'm 
>> mistaken there is no guarantee as of when it will be called, and some 
>> objects may have already been released (at lease I've had trouble in the 
>> past accessing certain objects from inside __del__, probably since the 
>> parent class's __del__ has to be called first, then it's objects are 
>> already released by the time I need to do something with them). Another 
>> method would be to implement something using the signal module and have 
>> a callback that does all the cleanup when the app. is 
>> quit/terminated/interrupted and have all the child classes override that 
>> with their cleanup code.
>>
>> What is the community's point of view on the subject?
>>
>> Thanks,
>> Gabriel
> 
> atexit? <http://docs.python.org/lib/module-atexit.html>
> 
> If it's only small things, there's try...finally, of course..

Huh, I totally didn't know that atexit is only run on normal
terminations. Oops. Never mind.
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Destructor?

[Matt Nordhoff]

Gabriel Rossetti wrote:
> Hello everyone,
> 
> we are writing an application that needs some cleanup to be done if the 
> application is quit, normally (normal termination) or by a signal like 
> SIGINT or SIGTERM. I know that the __del__ method exists, but unless I'm 
> mistaken there is no guarantee as of when it will be called, and some 
> objects may have already been released (at lease I've had trouble in the 
> past accessing certain objects from inside __del__, probably since the 
> parent class's __del__ has to be called first, then it's objects are 
> already released by the time I need to do something with them). Another 
> method would be to implement something using the signal module and have 
> a callback that does all the cleanup when the app. is 
> quit/terminated/interrupted and have all the child classes override that 
> with their cleanup code.
> 
> What is the community's point of view on the subject?
> 
> Thanks,
> Gabriel

atexit? 

If it's only small things, there's try...finally, of course..
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie: How to pass a dictionary to a function?

[Matt Nordhoff]

BonusOnus wrote:
> How do I pass a dictionary to a function as an argument?
> 
> 
> # Say I have a function foo...
> def foo (arg=[]):
> x = arg['name']
> y = arg['len']
> 
> s = len (x)
> 
> t = s + y
> 
> return (s, t)

I assume you actually indented the body of the function?

> # The dictionary:
> 
> dict = {}
> dict['name'] = 'Joe Shmoe'
> dict['len'] = 44

'dict' is the name of a built-in type. You should name your variable
something else.

> # I try to pass the dictionary as an argument to a
> # function
> 
> len, string = foo (dict)

'len' is the name of a built-in function and 'string' is a module in the
standard library. You should name both of them something else.

> # This bombs with 'TypeError: unpack non-sequence'
> 
> What am I doing wrong with the dictionary?

It would be helpful to provide the full traceback, since that says what
line the problem is on...

HTH (it probably won't)
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help replacing os.system call with subprocess call

[Matt Nordhoff]

David Pratt wrote:
> Hi David and Matt. I appreciate your help which has got me moving
> forward again so many thanks for your reply. I have been using
> subprocess.Popen a fair bit but this was the first time I had to use
> subprocess to capture large file output. The trouble I was having was
> with the process would just hang. Chunking was the solution. I guess I
> assumed this would be taken care of in the internals.
> 
> Overall, I wish subprocess had some better documentation since it is
> definitely not a drop in replacement for os.system. In other
> circumstances I am using subprocess.call() for simple calls which works
> fine.
> 
> The speed of this solution is slower than os.system. Would a queue of
> some kind be needed to speed this up? Has anyone implemented something
> like this? Many thanks.
> 
> Regards,
> David

Did you see my second message? That should help performance. If not, I'm
totally out of my depth and have no idea at all. Sorry.

(How much slower? 10%? 200%?)
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help replacing os.system call with subprocess call

[Matt Nordhoff]

Matt Nordhoff wrote:
> David Pratt wrote:
>> Hi. I am trying to replace a system call with a subprocess call. I have 
>> tried subprocess.Popen and subprocess.call with but have not been 
>> successful. The command line would be:
>>
>> svnadmin dump /my/repository > svndump.db
>>
>> This is what I am using currently:
>>
>> os.system('svnadmin dump %s > %s' % (svn_dir,
>>  os.path.join(backup_dir, 'svndump.db')))
>>
>> Many thanks.
> 
> Try this:
> 
> import os.path
> import subprocess
> 
> p = subprocess.Popen(
> ['svnadmin', 'dump', svndir],
> stdout=subprocess.PIPE,
> )
> 
> fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb')
> 
> while True:
> chunk = p.stdout.read(2**20) # 1 MB
> if not chunk:
> break
> fh.write(chunk)
> 
> fh.close()
> 
> It reads svnadmin's stdout in 1 MB chunks, in case it's large enough
> that reading the whole thing into RAM at once would be a bad idea.
> 
> No error handling. For one, you might want to add a try...finally to
> ensure that fh will get closed. (Or if you have Python 2.5, use a with
> statement! :-) ) Also, Popen will raise an OSError if svnadmin can't be
> found or something. And this isn't even considering svnadmin erroring out...
> 
> svnadmin's stderr will go to your stderr.
> 
> I didn't test it, but I'm pretty sure it will work. (I spotted a syntax
> error while writing that though.) I don't have much experience with
> Popen's stdio objects, so it's possible you'd need to do something like
> call p.wait() to wait for it to exit before being able to read its stdout.
> 
> It could be slower than the os.system version, since now Python is doing
> all of the I/O, instead of your shell, but I doubt that'll be a big problem.
> 
> (Also, insert suggestion about using a good VCS. ;-) )

This might work too:

import os.path
import subprocess

fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb')

p = subprocess.Popen(
['svnadmin', 'dump', svndir],
stdout=fh,
)

# Wait for svnadmin to exit so that fh can be closed
p.wait()
fh.close()

With this, svnadmin's stdout is directly set to the file. Python isn't
being an intermediary reading from stdout and writing it to the file. At
least, that's how I expect it would work.

I didn't test this either. In particular, if Popen doesn't behave how I
expect it should, it might buffer the entirety of stdout in memory or
something, which would be bad if your svn repo is really large...
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help replacing os.system call with subprocess call

[Matt Nordhoff]

David Pratt wrote:
> Hi. I am trying to replace a system call with a subprocess call. I have 
> tried subprocess.Popen and subprocess.call with but have not been 
> successful. The command line would be:
> 
> svnadmin dump /my/repository > svndump.db
> 
> This is what I am using currently:
> 
> os.system('svnadmin dump %s > %s' % (svn_dir,
>  os.path.join(backup_dir, 'svndump.db')))
> 
> Many thanks.

Try this:

import os.path
import subprocess

p = subprocess.Popen(
['svnadmin', 'dump', svndir],
stdout=subprocess.PIPE,
)

fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb')

while True:
chunk = p.stdout.read(2**20) # 1 MB
if not chunk:
break
fh.write(chunk)

fh.close()

It reads svnadmin's stdout in 1 MB chunks, in case it's large enough
that reading the whole thing into RAM at once would be a bad idea.

No error handling. For one, you might want to add a try...finally to
ensure that fh will get closed. (Or if you have Python 2.5, use a with
statement! :-) ) Also, Popen will raise an OSError if svnadmin can't be
found or something. And this isn't even considering svnadmin erroring out...

svnadmin's stderr will go to your stderr.

I didn't test it, but I'm pretty sure it will work. (I spotted a syntax
error while writing that though.) I don't have much experience with
Popen's stdio objects, so it's possible you'd need to do something like
call p.wait() to wait for it to exit before being able to read its stdout.

It could be slower than the os.system version, since now Python is doing
all of the I/O, instead of your shell, but I doubt that'll be a big problem.

(Also, insert suggestion about using a good VCS. ;-) )
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How easy is it to install python as non-root user?

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> Does python install fairly easily for a non-root user?
> 
> I have an ssh login account onto a Linux system that currently
> provides Python 2.4.3 and I'd really like to use some of the
> improvements in Python 2.5.x.
> 
> So, if I download the Python-2.5.2.tgz file is it just the standard:-
> 
> ./configure --prefix=$HOME
> make
> make install

It is easy, but on an oldish Debian box, I ran into problems where the
headers for some libraries weren't available, so my Python install
didn't support readline or bzip2 (or perhaps other things too).
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Strange loop behavior

[Matt Nordhoff]

Gabriel Rossetti wrote:
> Hello,
> 
> I wrote a program that reads data from a file and puts it in a string, 
> the problem is that it loops infinitely and that's not wanted, here is 
> the code :
> 
> d = repr(f.read(DEFAULT_BUFFER_SIZE))
> while d != "":
> file_str.write(d)
> d = repr(f.read(DEFAULT_BUFFER_SIZE))

FWIW, you could do it like this to avoid duplicating that line of code:

while True:
d = f.read(DEFAULT_BUFFER_SIZE)
if not d:
break
file_str.write(d)

Some people would also compress the whitespace a bit:

while True:
d = f.read(DEFAULT_BUFFER_SIZE)
if not d: break
file_str.write(d)



(I'll comment on your Unicode issues in a minute.)
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with PARAGRAPH SEPARATOR

[Matt Nordhoff]

[EMAIL PROTECTED] wrote:
> Actually that's what I tried to do, for example:
> outputString = myString.encode('iso-8859-1','ignore')
> 
> However, I always get such messages (the character, that's causing problems 
> varies, but its always higher than 127 ofc...)
> 
> 'ascii' codec can't decode byte 0xc3 in position 51: ordinal not in 
> range(128) 
> 
> It doesn't matter what encoding I use (tried 'utf-8' 'utf-16' 'latin_1' and 
> the iso one). The debugger is showing all the special characters (from french 
> and german language) so I'm wondering why there's still the message about the 
> 'ascii' codec...
> 
> Would that mean that the string "myString" is an ascii-string or what?

That's a *decode* error. If myString is a str object, it has to be
*decoded* before it can be *encoded* to ISO-8859-1 or whatever you want,
and it's defaulting to ASCII.

This page may help:

-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Generators

[Matt Nordhoff]

mpc wrote:



> def concatenate(sequences):
> for seq in sequences:
> for item in seq:
> yield item

You should check out itertools.chain(). It does this. You call it like
"chain(seq1, seq2, ...)" instead of "chain(sequences)" though, which may
be a problem for you.

The rest of itertools might be interesting too:




-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode/UTF-8 confusion

[Matt Nordhoff]

Tom Stambaugh wrote:
> I'm still confused about this, even after days of hacking at it. It's
> time I asked for help. I understand that each of you knows more about
> Python, Javascript, unicode, and programming than me, and I understand
> that each of you has a higher SAT score than me. So please try and be
> gentle with your responses.
>  
> I use simplejson to serialize html strings that the server is delivering
> to a browser. Since the apostrophe is a string terminator in javascript,
> I need to escape any apostrophe embedded in the html.
>  
> Just to be clear, the specific unicode character I'm struggling with is
> described in Python as:
> u'\N{APOSTROPHE}'}. It has a standardized utf-8 value (according to, for
> example, http://www.fileformat.info/info/unicode/char/0027/index.htm)
> of  0x27.
>  
> This can be expressed in several common ways:
> hex: 0x27
> Python literal: u"\u0027"
>  
> Suppose I start with some test string that contains an embedded
> apostrophe -- for example: u"   '   ". I believe that the appropriate
> json serialization of this is (presented as a list to eliminate notation
> ambiguities):
>  
> ['"', ' ', ' ', ' ', '\\', '\\', '0', '0', '2', '7', ' ', ' ', ' ', '"']
>  
> This is a 14-character utf-8 serialization of the above test string.
>  
> I know I can brute-force this, using something like the following:
> def encode(aRawString):
> aReplacement = ''.join(['\\', '0', '0', '2', '7'])
> aCookedString = aRawString.replace("'", aReplacement)
> answer = simplejson.dumps(aCookedString)
> return answer
>  
> I can't even make mailers let me *TYPE* a string literal for the
> replacement string without trying to turn it into an HTML link!
>  
> Anyway, I know that my "encode" function works, but it pains me to add
> that "replace" call before *EVERY* invocation of the simplejson.dumps()
> method. The reason I upgraded to 1.7.4 was to get the c-level speedup
> routine now offered by simplejson -- yet the need to do this apostrophe
> escaping seems to negate this advantage! Is there perhaps some
> combination of dumps keyword arguments, python encode()/str() magic, or
> something similar that accomplishes this same result?
>  
> What is the highest-performance way to get simplejson to emit the
> desired serialization of the given test string?

simplejson handles all necessary escaping of stuff like quotes...
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: request for Details about Dictionaries in Python

[Matt Nordhoff]

Michael Wieher wrote:
> I'm not sure if a well-written file/seek/read algorithm is faster than a
> relational database...
> sure a database can store relations and triggers and all that, but if
> he's just doing a lookup for static data, then I'm thinking disk IO is
> faster for him?  not sure

I would think that rolling your own solution would get complicated
enough that it would be easier to just use SQLite or something. If you
wanted to go to the effort, you could probably get something faster, but
if SQLite is fast enough, who cares?

Hmm, if Perl's on-disk hash tables are good, maybe someone should port
them to Python, or maybe someone already has.

Also, for translations, maybe there's a good library already available.
-- 
-- 
http://mail.python.org/mailman/listinfo/python-list