Evaluate my first python script, please

2010-03-04 Thread Pete Emerson
I've written my first python program, and would love suggestions for
improvement.

I'm a perl programmer and used a perl version of this program to guide
me. So in that sense, the python is "perlesque"

This script parses /etc/hosts for hostnames, and based on terms given
on the command line (argv), either prints the list of hostnames that
match all the criteria, or uses ssh to connect to the host if the
number of matches is unique.

I am looking for advice along the lines of "an easier way to do this"
or "a more python way" (I'm sure that's asking for trouble!) or
"people commonly do this instead" or "here's a slick trick" or "oh,
interesting, here's my version to do the same thing".

I am aware that there are performance improvements and error checking
that could be made, such as making sure the file exists and is
readable and precompiling the regular expressions and not calculating
how many sys.argv arguments there are more than once. I'm not hyper
concerned with performance or idiot proofing for this particular
script.

Thanks in advance.


#!/usr/bin/python

import sys, fileinput, re, os

filename = '/etc/hosts'

hosts = []

for line in open(filename, 'r'):
match = re.search('\d+\.\d+\.\d+\.\d+\s+(\S+)', line)
if match is None or re.search('^(?:float|localhost)\.', line):
continue
hostname = match.group(1)
count = 0
for arg in sys.argv[1:]:
for section in hostname.split('.'):
if section == arg:
count = count + 1
break
if count == len(sys.argv) - 1:
hosts.append(hostname)

if len(hosts) == 1:
os.system("ssh -A " + hosts[0])
else:
print '\n'.join(hosts)
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Evaluate my first python script, please

2010-03-04 Thread Pete Emerson
Great responses, thank you all very much. I read Jonathan Gardner's
solution first and investigated sets. It's clearly superior to my
first cut.

I love the comment about regular expressions. In perl, I've reached
for regexes WAY too much. That's a big lesson learned too, and from my
point of view not because of performance (although that's most likely
a bonus) but because of code readability.

Here's the version I have now. It looks quite similar to sjdevnull's.
Further comments appreciated, and thanks again.

#!/usr/bin/env python

import sys, fileinput, os

filename = '/etc/hosts'

hosts = []

search_terms = set(sys.argv[1:])

for line in open(filename, 'r'):
if line.startswith('#'): continue
try:
hostname = line.strip().split('\t')[2]  # The 
host I want is always
at the 3rd tab.
except IndexError:
continue
if hostname.startswith('float.') or
hostname.startswith('localhost.'):
continue
if search_terms <= set(hostname.split('.')):  # same as if
search_terms.issubset(hostname.split('.')):
hosts.append(hostname)

if len(hosts) == 1:
os.execl("/usr/bin/ssh", '-A', hosts[0])
else:
for host in hosts:
print host

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Evaluate my first python script, please

2010-03-05 Thread Pete Emerson
Thanks for your response, further questions inline.

On Mar 4, 11:07 am, Tim Wintle  wrote:
> On Thu, 2010-03-04 at 10:39 -0800, Pete Emerson wrote:
> > I am looking for advice along the lines of "an easier way to do this"
> > or "a more python way" (I'm sure that's asking for trouble!) or
> > "people commonly do this instead" or "here's a slick trick" or "oh,
> > interesting, here's my version to do the same thing".
>
> (1) I would wrap it all in a function
>
> def main():
>     # your code here
>
> if __name__ == "__main__":
>     main()

Is this purely aesthetic reasons, or will I appreciate this when I
write my own modules, or something else?

>
> (2) PEP8 (python style guidelines) suggests one import per line
>
> (3) I'd use four spaces as tab width
>
> (4)
> I'd change this:
>
> >     for arg in sys.argv[1:]:
> >         for section in hostname.split('.'):
> >             if section == arg:
> >                 count = count + 1
> >                 break
>
> to something more like:
>
>     for section in hostname.split("."):
>         if section in sys.argv[1:]:
>             count += 1

Ah, yes, I like that. It moves towards the set notation I've wound up
with. Definitely more readable to me.

>
> (although as you suggested I'd only calculate sys.argv[1:] once)
>
> ... or you could replace whole section between the for loop and
> hosts.append with:
>
>     if sorted(hostname.split(".")) == sorted(sys.argv[1:]):
>         host.append(hostname)

This doesn't actually work, because I'm not looking for a one to one
mapping of args to sections of hostname, but rather a subset. So
passing in 'prod sfo' would register a match for '001.webapp.prod.sfo'.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Evaluate my first python script, please

2010-03-05 Thread Pete Emerson
On Mar 5, 7:00 am, Duncan Booth  wrote:
> Jean-Michel Pichavant  wrote:
> > And tell me how not using regexp will ensure the /etc/hosts processing
> > is correct ? The non regexp solutions provided in this thread did not
> > handled what you rightfully pointed out about host list and commented
> > lines.
>
> It won't make is automatically correct, but I'd guess that written without
> being so dependent on regexes might have made someone point out those
> deficiencies sooner. The point being that casual readers of the code won't
> take the time to decode the regex, they'll glance over it and assume it
> does something or other sensible.
>
> If I was writing that code, I'd read each line, strip off comments and
> leading whitespace (so you can use re.match instead of re.search), split on
> whitespace and take all but the first field. I might check that the field
> I'm ignoring it something like a numeric ip address, but if I did want to
> do then I'd include range checking for valid octets so still no regex.
>
> The whole of that I'd wrap in a generator so what you get back is a
> sequence of host names.
>
> However that's just me. I'm not averse to regular expressions, I've written
> some real mammoths from time to time, but I do avoid them when there are
> simpler clearer alternatives.
>
> > And FYI, the OP pattern does match '192.168.200.1 (foo123)'
> > ...
> > Ok that's totally unfair :D You're right I made a mistake.  Still the
> > comment is absolutely required (provided it's correct).
>
> Yes, the comment would have been good had it been correct. I'd also go for
> a named group as that provides additional context within the regex.
>
> Also if there are several similar regular expressions in the code, or if
> they get too complex I'd build them up in parts. e.g.
>
> OCTET = r'(?:\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])'
> ADDRESS = (OCTET + r'\.') * 3 + OCTET
> HOSTNAME = r'[-a-zA-Z0-9]+(?:\.[-a-zA-Z0-9]+)*'
>   # could use \S+ but my Linux manual says
>   # alphanumeric, dash and dots only
> ... and so on ...
>
> which provides another way of documenting the intentions of the regex.
>
> BTW, I'm not advocating that here, the above patterns would be overkill,
> but in more complex situations thats what I'd do.
>
> --
> Duncan Boothhttp://kupuguy.blogspot.com

All good comments here. The takeaway for my lazy style of regexes
(which makes it harder for non-regex fiends to read, regardless of the
language) is that there are ways to make regexes much more readable to
the untrained eye. Duncan, I like your method of defining sections of
the regex outside the regex itself, even if it's a one time use.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Evaluate my first python script, please

2010-03-05 Thread Pete Emerson
On Mar 5, 10:19 am, "sjdevn...@yahoo.com"  wrote:
> On Mar 5, 10:53 am, Pete Emerson  wrote:
>
>
>
>
>
> > Thanks for your response, further questions inline.
>
> > On Mar 4, 11:07 am, Tim Wintle  wrote:
>
> > > On Thu, 2010-03-04 at 10:39 -0800, Pete Emerson wrote:
> > > > I am looking for advice along the lines of "an easier way to do this"
> > > > or "a more python way" (I'm sure that's asking for trouble!) or
> > > > "people commonly do this instead" or "here's a slick trick" or "oh,
> > > > interesting, here's my version to do the same thing".
>
> > > (1) I would wrap it all in a function
>
> > > def main():
> > >     # your code here
>
> > > if __name__ == "__main__":
> > >     main()
>
> > Is this purely aesthetic reasons, or will I appreciate this when I
> > write my own modules, or something else?
>
> Suppose the above code is in mymodule.py.  By wrapping main() you can:
> 1. Have another module do:
> import mymodule
> ... (so some stuff, perhaps munge sys.argv)
> mymodule.main()
> 2. If mymodule has a small function in it, someone else can import it
> and call that function
> 3. You can run pylint, pychecker and other source-code checkers that
> need to be able to import your module to check it (I wouldn't be
> surprised if recent versions of one or the other of those don't
> require imports, and some checkers like pyflakes certainly don't).
> 4. You can easily have a unit tester call into the module
>
> etc.
>
> > > (2) PEP8 (python style guidelines) suggests one import per line
>
> > > (3) I'd use four spaces as tab width
>
> +1 on both; it's good to get into the habit of writing standard-
> looking Python code.

Agreed, noted, and appreciated, with the caveat that using spaces
instead of tabs might border on an emacs vs. vi flamewar in some
circles. I personally will use spaces going forward.
-- 
http://mail.python.org/mailman/listinfo/python-list


Conditional based on whether or not a module is being used

2010-03-05 Thread Pete Emerson
In a module, how do I create a conditional that will do something
based on whether or not another module has been loaded?

Suppose I have the following:

import foo
import foobar

print foo()
print foobar()

### foo.py
def foo:
return 'foo'

### foobar.py
def foobar:
if foo.has_been_loaded(): # This is not right!
return foo() + 'bar'  # This might need to be foo.foo() ?
else:
return 'bar'

If someone is using foo module, I want to take advantage of its
features and use it in foobar, otherwise, I want to do something else.
In other words, I don't want to create a dependency of foobar on foo.

My failed search for solving this makes me wonder if I'm approaching
this all wrong.

Thanks in advance,
Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Conditional based on whether or not a module is being used

2010-03-05 Thread Pete Emerson
On Mar 5, 11:24 am, Pete Emerson  wrote:
> In a module, how do I create a conditional that will do something
> based on whether or not another module has been loaded?
>
> Suppose I have the following:
>
> import foo
> import foobar
>
> print foo()
> print foobar()
>
> ### foo.py
> def foo:
>     return 'foo'
>
> ### foobar.py
> def foobar:
>     if foo.has_been_loaded(): # This is not right!
>         return foo() + 'bar'      # This might need to be foo.foo() ?
>     else:
>         return 'bar'
>
> If someone is using foo module, I want to take advantage of its
> features and use it in foobar, otherwise, I want to do something else.
> In other words, I don't want to create a dependency of foobar on foo.
>
> My failed search for solving this makes me wonder if I'm approaching
> this all wrong.
>
> Thanks in advance,
> Pete

Aha, progress. Comments appreciated. Perhaps there's a different and
more conventional way of doing it than this?

def foobar():
import sys
if 'foomodule' in sys.modules.keys():
import foomodule
return foomodule.foo() + 'bar'
else:
return 'bar'
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Conditional based on whether or not a module is being used

2010-03-05 Thread Pete Emerson
On Fri, Mar 5, 2010 at 12:17 PM, Chris Rebert  wrote:
> On 3/5/10, Pete Emerson  wrote:
>> In a module, how do I create a conditional that will do something
>> based on whether or not another module has been loaded?
>>
>> Suppose I have the following:
>>
>> import foo
>> import foobar
>>
>> print foo()
>> print foobar()
>>
>> ### foo.py
>> def foo:
>>    return 'foo'
>>
>> ### foobar.py
>> def foobar:
>>    if foo.has_been_loaded(): # This is not right!
>>        return foo() + 'bar'      # This might need to be foo.foo() ?
>>    else:
>>        return 'bar'
>>
>> If someone is using foo module, I want to take advantage of its
>> features and use it in foobar, otherwise, I want to do something else.
>> In other words, I don't want to create a dependency of foobar on foo.
>>
>> My failed search for solving this makes me wonder if I'm approaching
>> this all wrong.
>
> Just try importing foo, and then catch the exception if it's not installed.
>
> #foobar.py
> try:
>    import foo
> except ImportError:
>    FOO_PRESENT = False
> else:
>    FOO_PRESENT = True
>
> if FOO_PRESENT:
>    def foobar():
>        return foo.foo() + 'bar'
> else:
>    def foobar():
>        return 'bar'
>
>
> You could alternately do the `if FOO_PRESENT` check inside the
> function body rather than defining separate versions of the function.
>
> Cheers,
> Chris
> --
> http://blog.rebertia.com
>

Except I want to use the module only if the main program is using it
too, not just if it's available for use. I think that I found a way in
my follow-up post to my own message, but not sure it's the best way or
conventional.

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Conditional based on whether or not a module is being used

2010-03-05 Thread Pete Emerson
On Mar 5, 12:06 pm, "Martin P. Hellwig" 
wrote:
> On 03/05/10 19:24, Pete Emerson wrote:
>
> > In a module, how do I create a conditional that will do something
> > based on whether or not another module has been loaded?
> >
> > If someone is using foo module, I want to take advantage of its
> > features and use it in foobar, otherwise, I want to do something else.
> > In other words, I don't want to create a dependency of foobar on foo.
>
> > My failed search for solving this makes me wonder if I'm approaching
> > this all wrong.
>
> > Thanks in advance,
> > Pete
>
> Hmm how about the module is available, just not imported yet, I would
> assume that you still would like to use the module then.
> Perhaps playing around with the imp module might get you what you mean
> instead of what you say?
>
> --
> mph

I can certainly see why one might want to use it if it's available but
not yet imported. In that case I could do a try / exception block. But
in this case, I actually don't want to use the module unless the main
program is doing it too. But you've got me thinking, I need to make
sure that's really the desired behavior.

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Conditional based on whether or not a module is being used

2010-03-05 Thread Pete Emerson
On Mar 5, 11:57 am, MRAB  wrote:
> Pete Emerson wrote:
> > In a module, how do I create a conditional that will do something
> > based on whether or not another module has been loaded?
>
> > Suppose I have the following:
>
> > import foo
> > import foobar
>
> > print foo()
> > print foobar()
>
> > ### foo.py
> > def foo:
> >     return 'foo'
>
> > ### foobar.py
> > def foobar:
> >     if foo.has_been_loaded(): # This is not right!
> >         return foo() + 'bar'      # This might need to be foo.foo() ?
> >     else:
> >         return 'bar'
>
> > If someone is using foo module, I want to take advantage of its
> > features and use it in foobar, otherwise, I want to do something else.
> > In other words, I don't want to create a dependency of foobar on foo.
>
> > My failed search for solving this makes me wonder if I'm approaching
> > this all wrong.
>
> Look for its name in sys.modules, for example:
>
>      'foo' in sys.modules

Excellent, this is what I finally discovered, although I was looking
for 'foo' in sys.modules.keys(), which apparently isn't necessary.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Conditional based on whether or not a module is being used

2010-03-05 Thread Pete Emerson
On Mar 5, 1:14 pm, Chris Rebert  wrote:
> On Fri, Mar 5, 2010 at 12:25 PM, Pete Emerson  wrote:
> > On Fri, Mar 5, 2010 at 12:17 PM, Chris Rebert  wrote:
> >> On 3/5/10, Pete Emerson  wrote:
> >>> In a module, how do I create a conditional that will do something
> >>> based on whether or not another module has been loaded?
> 
> >>> If someone is using foo module, I want to take advantage of its
> >>> features and use it in foobar, otherwise, I want to do something else.
> >>> In other words, I don't want to create a dependency of foobar on foo.
>
> >>> My failed search for solving this makes me wonder if I'm approaching
> >>> this all wrong.
>
> >> Just try importing foo, and then catch the exception if it's not installed.
> 
> > Except I want to use the module only if the main program is using it
> > too, not just if it's available for use. I think that I found a way in
> > my follow-up post to my own message, but not sure it's the best way or
> > conventional.
>
> What is your use case for this behavior exactly? You've piqued my curiosity.
>
> Cheers,
> Chris
> --http://blog.rebertia.com

I have written my first module called "logger" that logs to syslog via
the syslog module but also allows for logging to STDOUT in debug mode
at multiple levels (to increase verbosity depending on one's need), or
both. I've looked at the logging module and while it might suit my
needs, it's overkill for me right now (I'm still *very* much a python
newbie).

I want to write other modules, and my thinking is that it makes sense
for those modules to use the "logger" module to do the logging, if and
only if the parent using the other modules is also using the logger
module.

In other words, I don't want to force someone to use the "logger"
module just so they can use my other modules, even if the "logger"
module is installed ... but I also want to take advantage of it if I'm
using it.

Now that I've written that, I'm not sure that makes a whole lot of
sense. It seems like I could say, "hey, this person has the 'logger'
module available, let's use it!".

Thoughts?
-- 
http://mail.python.org/mailman/listinfo/python-list


best practices: is collections.defaultdict my friend or not?

2010-03-05 Thread Pete Emerson
I've been wrestling with dicts. I hope at the very least what I
discovered helps someone else out, but I'm interested in hearing from
more learned python users.

I found out that adding a two dimensional element without defining
first dimension existing doesn't work:

>>> data = {}
>>> data['one']['two'] = 'three'
Traceback (most recent call last):
  File "", line 1, in 
KeyError: 'one'
>>> data['one'] = {}
>>> data['one']['two'] = 'three'
>>> print data
{'one': {'two': 'three'}}

And through some research, I discovered collections.defaultdict (new
in Python 2.5, FWIW):

>>> import collections
>>> data = collections.defaultdict(dict)
>>> data['one']['two'] = 'three'
>>> print data
defaultdict(, {'one': {'two': 'three'}})

Why isn't the behavior of collections.defaultdict the default for a
dict?
Am I just revelling in my bad perl habits by not wanting to declare a
previous level first?
Is this sort of "more rigid" way of doing things common throughout
python, and is it best that I not fight it, but embrace it?

Your thoughts and comments are very much appreciated. I think my brain
already knows some of the answers, but my heart ... well, perl and I
go way back. Loving python so far, though.

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: best practices: is collections.defaultdict my friend or not?

2010-03-05 Thread Pete Emerson
On Mar 5, 6:10 pm, Andreas Waldenburger 
wrote:
> On Fri, 5 Mar 2010 17:22:14 -0800 (PST) Pete Emerson
>
>
>
>
>
>  wrote:
> > [snip]
> > >>> data['one'] = {}
> > >>> data['one']['two'] = 'three'
> > >>> print data
> > {'one': {'two': 'three'}}
>
> > And through some research, I discovered collections.defaultdict (new
> > in Python 2.5, FWIW):
>
> > >>> import collections
> > >>> data = collections.defaultdict(dict)
> > >>> data['one']['two'] = 'three'
> > >>> print data
> > defaultdict(, {'one': {'two': 'three'}})
>
> > [snip]
> > Your thoughts and comments are very much appreciated. I think my brain
> > already knows some of the answers, but my heart ... well, perl and I
> > go way back. Loving python so far, though.
>
> Oh, by the way: That defaultdict route is a pretty solid solution. Not
> sure what problem you're trying to solve -- depending on your usecase,
> there might be a better approach.
>
> If you're just asking hypothetically and you're trying to apply a
> Perl idiom to Python, there probably *is* a better solution.
>
> /W
>
> --
> INVALID? DE!

I found out about the need to declare the higher level as I was
reading in a JSON struct into a dict and then adding a new entry at a
lower level. Mostly just proof of concept stuff as I'm learning
python. I'm not sure that the use of defaultdict is really warranted
for me anywhere just yet. Mostly, I don't want to convert my perl to
python, that seems very counterproductive. Thank you very much for
your insight.

I was a little frightened of doing "import this" ("Hey, kid, run rm -
rf / and see what happens!"), but did, and the words are wise. :)

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: best practices: is collections.defaultdict my friend or not?

2010-03-05 Thread Pete Emerson
On Mar 5, 8:24 pm, Steven D'Aprano  wrote:
> On Fri, 05 Mar 2010 17:22:14 -0800, Pete Emerson wrote:
> > Why isn't the behavior of collections.defaultdict the default for a
> > dict?
>
> Why would it be?
>
> If you look up a key in a dict:
>
> addressbook['Barney Rubble']
>
> and you don't actually have Barney's address, should Python guess and
> make something up?
>
> In general, looking up a missing key is an error, and errors should never
> pass silently unless explicitly silenced.
>
> And for those cases where missing keys are not errors, you're spoiled for
> choice:
>
> dict.get
> dict.setdefault
> collections.defaultdict
>
> try:
>     dict[key]
> except KeyError:
>     do something else
>
> Or even:
>
> if key in dict:
>     dict[key]
> else:
>     do something else
>
> --
> Steven

My frame of reference for the past 10 < N < 15 years has been doing
this sort of assignment in perl:

$hash{key1}{key2} = value

I definitely agree that looking up a missing key should give an error.
The lazy perl programmer in me just wants to make an assignment to a
missing second key without defining the first key first. I'm not
saying it's right, I'm saying that it's something I'm trying to
unlearn, as I'm being convinced that it's the "best way" to do it in
python.

I'll take a look at dict.get and dict.setdefault, thank you for those.

I'm learning, and you're all very helpful, and I appreciate it!

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: best practices: is collections.defaultdict my friend or not?

2010-03-05 Thread Pete Emerson
On Mar 5, 6:26 pm, MRAB  wrote:
> Pete Emerson wrote:
> > I've been wrestling with dicts. I hope at the very least what I
> > discovered helps someone else out, but I'm interested in hearing from
> > more learned python users.
>
> > I found out that adding a two dimensional element without defining
> > first dimension existing doesn't work:
>
> >>>> data = {}
> >>>> data['one']['two'] = 'three'
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > KeyError: 'one'
> >>>> data['one'] = {}
> >>>> data['one']['two'] = 'three'
> >>>> print data
> > {'one': {'two': 'three'}}
>
> > And through some research, I discovered collections.defaultdict (new
> > in Python 2.5, FWIW):
>
> >>>> import collections
> >>>> data = collections.defaultdict(dict)
> >>>> data['one']['two'] = 'three'
> >>>> print data
> > defaultdict(, {'one': {'two': 'three'}})
>
> > Why isn't the behavior of collections.defaultdict the default for a
> > dict?
> > Am I just revelling in my bad perl habits by not wanting to declare a
> > previous level first?
> > Is this sort of "more rigid" way of doing things common throughout
> > python, and is it best that I not fight it, but embrace it?
>
> > Your thoughts and comments are very much appreciated. I think my brain
> > already knows some of the answers, but my heart ... well, perl and I
> > go way back. Loving python so far, though.
>
> Someone once wrote about a case where he was porting a frontend from
> Perl to Python. It called a backend and parsed the result. Sometimes
> converting one of the fields to a number would raise a ValueError
> because it would contain "ERR" instead of a number, which Perl, of
> course, would silently convert to 0!
>
> Python is all about refusing to guess, and complaining if there's an
> error. :-)

Perl is quite an amazing language, but it also definitely allows for
sloppy habits and poor coding behaviors, particularly for someone such
as me whose perl is completely self taught. I think that's why this
time around with python I'm trying to learn my prior experience and
seek help in areas where I suspect there is improvement to be made.
I'm really liking the rigid flexibility I'm experiencing with python
so far.

I'm thinking it's time for me to get a python reference or two, just
to kill a few trees. If anyone has any strong feelings about what
belongs in a "beginner but already learning quickly" library, please
toss them my way. I'll grep around the 'net for opinions on it, too.

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Conditional based on whether or not a module is being used

2010-03-06 Thread Pete Emerson
On Mar 6, 2:38 pm, Vinay Sajip  wrote:
> On Mar 5, 9:29 pm, Pete Emerson  wrote:
>
>
>
> > I have written my first module called "logger" that logs to syslog via
> > the syslog module but also allows forloggingto STDOUT in debug mode
> > at multiple levels (to increase verbosity depending on one's need), or
> > both. I've looked at theloggingmodule and while it might suit my
> > needs, it's overkill for me right now (I'm still *very* much a python
> > newbie).
>
> Overkill in what sense? You just need to write a few lines of code to
> be able to use the logging package which comes with Python:
>
> import logging, logging.handlers, sys
> logging.basicConfig(level=logging.DEBUG, stream=sys.stdout)
> logging.getLogger().addHandler(logging.handlers.SysLogHandler())
> # default logs to syslog at (localhost, 514) with facility LOG_USER
> # you can change the default to use e.g. Unix domain sockets and a
> different facility
>
> So you're experienced enough and have time enough to write your own
> logger module, but too much of a newbie to use a module which is part
> of Python's included batteries? If you're writing something like
> logging to learn about it and what the issues are, that's fair enough.
> But I can't see what you mean by overkill, exactly. The three lines
> above (or thereabouts) will, I believe, let you log to syslog and to
> stdout...which is what you say you want to do.
>
> > I want to write other modules, and my thinking is that it makes sense
> > for those modules to use the "logger" module to do thelogging, if and
> > only if the parent using the other modules is also using the logger
> > module.
>
> > In other words, I don't want to force someone to use the "logger"
> > module just so they can use my other modules, even if the "logger"
> > module is installed ... but I also want to take advantage of it if I'm
> > using it.
>
> > Now that I've written that, I'm not sure that makes a whole lot of
> > sense. It seems like I could say, "hey, this person has the 'logger'
> > module available, let's use it!".
>
> > Thoughts?
>
> Well, the logging package is available in Python and ready for use and
> pretty much battle tested, so why not use that? Are you planning to
> use third-party libraries in your Python work, or write everything
> yourself? If you are planning to use third party libraries, how would
> their logging be hooked into your logger module? And if not, is it
> good to have two logging systems in parallel?
>
> Of course as the maintainer of Python's logging package, you'd expect
> me to be biased in favour of it. You maybe shouldn't let that sway
> you ;-)
>
> Regards,
>
> Vinay Sajip

Thanks for your insights, Vinay, and thank you also for writing
packages such as logging. The word 'overkill' was a poor choice on my
part! I should have said, "I don't quite understand the logging module
yet, but I am comfortable with the syslog module's two functions,
openlog and syslog".

I wrote my own logger module *partly* to gain the experience, and
partly to do the following:

1) In debug mode, send what would have gone to syslog to STDOUT or
STDERR
2) In non-debug mode, use /dev/log or localhost:514 depending on what
is set
3) Allow for multiple levels of logging beyond INFO, WARNING, CRIT ...
essentially allow multiple levels of INFO depending on how much detail
is desired.  A high level of messaging when programs are running
poorly is desired, but when programs are running smoothly, I don't
need to send as much to syslog.

I started in with your logging package, but I think I simply got ahead
of myself. I definitely agree that writing my own wrappers around
syslog to do what I want might be a duplication of effort. At this
point I think I'm ready to go back to your logging package and see
what I can do; if you have words of advice regarding 1-3 above, I'd
certainly appreciate it.

Now I'll go to your example above and see what it does. Thank you!

Pete
-- 
http://mail.python.org/mailman/listinfo/python-list


Exiting gracefully from ThreadingTCPServer

2010-03-12 Thread Pete Emerson
I'm trying to get threading going for the first time in python, and
I'm trying to modify code I found so that I can have the server close
the TCP connections and exit gracefully. Two problems:

1) While the KeyboardInterrupt works, if I make more than 0 curls to
the server and then quit, I can't run it again right away and get
this:
socket.error: [Errno 48] Address already in use

Not all of my connections are closing properly. How do I fix this?

2) curling localhost:8080/quit does show the "Quitting" output that I
expect, but doesn't quit the server until I manually control-c it.

I think that I need *all* threads to close and not just the current
one, so I'm not quite sure how to proceed. Pointers in the right
direction are appreciated. And if there's a "better" way to do this
threading httpd server (subjective, I realize), please let me know!
Thanks.

Pete

#
#!/usr/bin/env python

import SocketServer
import SimpleHTTPServer

PORT = 8080
done = False
class CustomHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
def do_GET(self):
global done
if self.path=='/quit':
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write('Quitting')
done = True
return self
else:
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write('Unknown')
return self


if __name__ == "__main__":
httpd = SocketServer.ThreadingTCPServer(('localhost',
PORT),CustomHandler)
try:
while not done:
print "done: ", done
httpd.handle_request()
except KeyboardInterrupt:
print "Server is done."
httpd.server_close()
-- 
http://mail.python.org/mailman/listinfo/python-list