ANNOUNCE: awstats_reader 0.5

2009-12-19 Thread Joshua Kugler
ABOUT THE MODULE

AwstatsReader is a pythonic interface to AWStats data cache files.  Using
it, you can access year and month via dict-like subscripts, and and
individual data points via both dict-like subscripts and attribute-like
accessors.

As of version 0.5, it includes a script for merging AWStats Cache files.

Download: http://azariah.com/open_source.html

ABOUT THE AUTHOR

Joshua Kugler (jos...@azariah.com) is a programmer and system administator
with over 10 years of industory experience.  He is currently looking for a
job.  Happen to have one you could offer him? :)
Resume at: http://jjncj.com/papers/KuglerResume.pdf

DISCLAIMER
==
This is an early-beta release.  There are 43 tests which cover most, if not
all of the functionality, but not much documentation.  The interface should
be considered stable, but not in concrete.  The usage of this project in
a "real world" situation (awstats_cache_merge.py) led to many improvements
to the API.

I wrote this via examples from an AWStats cache file, so I'm sure there are
sections for which I do not have definitions.  If you would send me those
sections, I'll be sure to add them.

Right now, this will parse and display cache files from AWStats 6.5. I've
not tested other versions yet, as 6.5 is the only version I've had access
to so far.

REPOSITORY
==
No public repository yet. Just haven't set it up.

LICENSE
===
Modified BSD

EXAMPLE
===
import awstats_reader

obj  = awstats_reader.AwstatsReader('/path/to/awstats_logs', 'example.com')

print obj[2007]
print obj[2008][6]
m = obj[2009][7]
print m['general']
# Access like a dictionary...
print m['general']['LastLine']
#...or like an object attribute
print m['general'].LastLine
print m.general.LastLine

FEEDBACK

Please send questions/comments/suggestions to awstatsrea...@azariah.com
For now, you can find the latest version here:
http://azariah.com/open_source.html


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: IDE+hg

2009-11-23 Thread Joshua Kugler
NiklasRTZ wrote:
> If you
> know
> a good light IDE with hg, please inform. Topic handled earlier, still
> undecided
> http://groups.google.com/group/google-appengine-python/browse_thread/...
> Thanks in advance
> Niklas Rosencrantz

WingIDE support Hg, as well as svn, git, and many others.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: What IDE has good git and python support?

2009-10-27 Thread Joshua Kugler
Aweks wrote:
> what do you use?

WingIDE has excellent Python support (it's a Python IDE, after all), and the
latest version has full support for git.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Is pythonic version of scanf() or sscanf() planned?

2009-10-09 Thread Joshua Kugler
ryniek90 wrote:
> So maybe someone, someday decide to
> put in Python an alternative, really great implementation of scanf() ?

My idea of a "great scanf() function" would be a clever combination of
re.match(), int(), and float().

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: sftp login without password

2009-10-06 Thread Joshua Kugler
David wrote:
> transport.connect(username = username, pkey = mykey)
> 
> I get a "AuthenticationException: Authentication failed." exception.
> 
> 
> My ~/.ssh/id_rsa is correct because if, at console, I type
> 
> bags...@bagvapp:~$ sftp bags...@192.168.92.129
> Connecting to 192.168.92.129...
> sftp>
> 
> I get a successful login. How can I get an sftp login without using a
> password in python?
> I am using Ubuntu 9.04, python 2.6.2 and paramiko 1.7.5

What is the contents of 'mykey'?  Is it a file object? A string containing
the contents of ~/.ssh/id_rsa? The Paramiko docs indicate mykey should be a
subclass of PKey, probably RSAKey.

So, you'd need to do:

mykey = RSAKey.from_private_key_file('/path/to/private/key/id_rsa')
transport.connect(username = username, pkey = mykey)

Beyond that, I"m not sure.  Of course, I've never used Paramiko before.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Do anyone here use Python *embedded* in a database?

2009-08-05 Thread Joshua Kugler
Jonathan Fine wrote:

> Hi
> 
> I'm writing a talk that compares embed and extend, and wondered if
> anyone here ever used the Python *embedded* in a database server.
>  http://twistedmatrix.com/users/glyph/rant/extendit.html
> 
> Web frameworks (such as Django) use extend, to import an extension
> module that makes a connection to a database.
> 
> If you have used embed, you might have consulted a page such as:
>  http://www.postgresql.org/docs/8.3/interactive/plpython-funcs.html
> 

Yes, I've used plpython.  We had a web application that had account creation
logic with salting, encryption, etc, on the passwords.  We then wanted to
use the same accounts to log in to a Drupal site.  Instead of trying to
replicate the logic in PHP, I just created a stored procedure with
plpython, and we overrode the drupal login check to call something
like "SELECT check_account(user, pass)" which would return 1 or 0 on
success or failure, respectively.  Worked great.

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Subclassing Python's dict

2009-08-05 Thread Joshua Kugler
Xavier Ho wrote:

> On Thu, Aug 6, 2009 at 11:51 AM, Sergey Simonenko
> wrote:
> 
>> I subclass builtin 'dict' in my application and experience some problems
>> with it.
>>
> 
> You should subclass collections.UserDict, and not the default dict class.
> Refer to the collections module.

Are you referring to Python 3.0?  Python 2.6 does not have
collections.UserDict

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: len() should always return something

2009-07-27 Thread Joshua Kugler
Dr. Phillip M. Feldman wrote:

> 
> "As far as I know, there is no programming language which treats scalars
> like ints as if they were
> vectors of length 1"
> 
> Actually, Matlab does:
> 
>>> length(5)
> ans =
>  1
>>> 

Oddly enough, so does Perl:

$ print length(44)
2

(that's in the Zoidberg shell)

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Opening a SQLite database in readonly mode

2009-07-09 Thread Joshua Kugler
Joshua Kugler wrote:
> Sorry about that...since pysqlite and APSW are both discusses on the
> pysqlite list, I had made an incorrect assumption. Oops.

"are both discusses?"  Yeef, I must have been out of it.  Discussed, thank
you. :)

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Opening a SQLite database in readonly mode

2009-07-07 Thread Joshua Kugler
Roger Binns wrote:
> Joshua Kugler wrote:
>> BTW, APSW is written by the same author as pysqlite.
> Not even remotely true :-)

Sorry about that...since pysqlite and APSW are both discusses on the
pysqlite list, I had made an incorrect assumption. Oops.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Opening a SQLite database in readonly mode

2009-07-06 Thread Joshua Kugler
Paul Moore wrote:
> The SQLite documentation mentions a flag, SQLITE_OPEN_READONLY, to
> open a database read only. I can't find any equivalent documented in
> the Python standard library documentation for the sqlite3 module (or,
> for that matter, on the pysqlite library's website).
> 
> Is it possible to open a sqlite database in readonly mode, in Python?

Yes, but most likely not with pysqlite.  The python sqlite3 module in the
standard library, and the pysqlite module are both DB-API compliant, which
means they probably do not have a method to open the DB read only (as that
is usually enforced at the user permission level).

If you want to use that flag, take a look at APSW. It is a very thin layer
on top of the SQLite C API, and thus supports everything that the C API
supports.  It is not, however, DB API compliant, but is very close so not
hard to pick up.

BTW, APSW is written by the same author as pysqlite.

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How does Python's OOP feel?

2009-05-28 Thread Joshua Kugler
> The good thing about python is : it 'tastes' like what it was being
> advertised

+1 QOTW


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Compile python extensions under windows/cygwin

2009-05-26 Thread Joshua Kugler
Joana wrote:

> I mantain Python on Windows, all installed packages are under c:
> \Python25\Lib\site-packages. Now I have to build C libraries used by
> python extensions and I am using cygwin, but I don't know how to
> install the module in Windows directory.

I have used MingW to do this.  Might this help:

http://joshuakugler.com/archives/13-Building-Python-Eggs-with-C-Extensions-on-Windows.html

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Why is it that *dbm modules don't provide an iterator? (Language design question)

2009-04-09 Thread Joshua Kugler
Akira Kitada wrote:

> The loop has to be:
> """
 k = d.firstkey()
 while k != None:
> ...print k
> ...k = d.nextkey(k)
> key2
> key1
> """

Why not

for key in d.keys():
print key

That worked for me.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: VMware and pywin32 error...

2009-03-13 Thread Joshua Kugler
dot wrote:
> has anyone experience with installing Python and pywin32 to Windows XP
> Pro running in a VMware environment?
> 
> At the end of installing pywin32 I get following error:
> 
> 
> Traceback (most recent call last):
>File "", line 601, in 
>File "", line 311, in install
>File "", line 149, in LoadSystemModule
> ImportError: DLL load failed: This application has failed to start
> because the application configuration is incorrect. Reinstalling the
> application may fix this problem.
> 
> 
> I use following versions:
> 
> VMware Workstation 6.5.0 build-118166
> Microsoft Windows XP Professional English Build 2600
> added SP1 and than SP3
> Python 2.6.1
> pywin32 213

I've installed and run Python and pywin32 in a VMWare environment and have
had no trouble.  This is not an issue with VMWare. Something is
misconfigured in your XP and/or Python install.  I'm assuming you installed
Python before installing pywin32?

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: "/a" is not "/a" ?

2009-03-06 Thread Joshua Kugler
Emanuele D'Arrigo wrote:

 c = "/a"
 d = "/a"
 c == d
> True   # all good so far
 c is d
> False # ek!
> 
> Why c and d point to two different objects with an identical string
> content rather than the same object?

Because you instantiated two difference objects.
http://docs.python.org/reference/datamodel.html#objects-values-and-types
should get you started on Python and objects.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: urllib2 login help

2009-02-23 Thread Joshua Kugler
You also might want to look at http://wwwsearch.sourceforge.net/mechanize/ 
It will do most of the hard stuff for you (sending form elements, keeping
track of cookies, etc.).

j

john.weather...@gmail.com wrote:

> Hello,
> 
> I'm having trouble using urllib2 (maybe) when trying to log into a web
> site that requires a user to enter a login name and a password
> (authentication).  I've tried many things but none seem to work and
> have become stuck recently and was hoping to get a hint from those
> much more knowledgeable than myself.
> 
> I want to automate logging on to the investopedia stock simulator web
> site.
> 
> http://simulator.investopedia.com/authorization/login.aspx
> 
> I can't seem to automate this successfully.
> 
> My python script is below:
> 
> import os,re
> import urllib, urllib2
> 
> theurl = "http://simulator.investopedia.com/authorization/login.aspx";
> 
> post_dict = { "ct100$MainPlaceHolder$usernameText": "XXX",
> "ct100$MainPlaceHolder$passwordText": "XXX", "ct100$MainPlaceHolder
> $loginButton": "Sign In", "ct100$MainPlaceHolder$rememberMeCheckBox":
> "on" }
> 
> post_data = urllib.urlencode( post_dict )
> 
> headers = { 'User-agent': 'Mozilla/4.0 (compatible; MSIE 5.5; Windows
> NT)',
> 'Host': 'simulator.investopedia.com',
> 'Referer': 'http://simulator.investopedia.com/
> authorization/login.aspx',
> }
> 
> req = urllib2.Request( theurl, post_data, headers )
> response = urllib2.urlopen(req)
> the_page = response.read()
> 
> The problem is that this page returned seems to be the same as the
> login page which I connected to initially.  When I perform the login
> process manually things work as expected.  What am I doing
> incorrectly?
> 
> *ANY* hints/suggestions/directions would be very appreciated since
> I've run out of ideas of things to try at this point.
> 
> Thanks very much
> 
> 
> --
> http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list


Re: Change in cgi module's handling of POST requests

2009-02-12 Thread Joshua Kugler
Bob Kline wrote:

> [Didn't realize the mirror didn't work both ways]
> 
> We just upgraded Python to 2.6 on some of our servers and a number of our
> CGI scripts broke because the cgi module has changed the way it handles
> POST requests.

> When the 'action' attribute was not present in the form 
> element on an HTML page the module behaved as if the value of the
> attribute was the URL which brought the user to the page with the form,
> but without the query (?x=y...) part.

This does not make sense.  Can you give an example?  When you POST, the
attributes are not appended to the URL, but form encoded.

> Now FieldStorage.getvalue () is 
> giving the script a list of two copies of the value for some of the
> parameters (folding in the parameters from the previous request) instead
> of the single string it used to return for each.

There is a function call to get only one value.  I think it's get_first() or
some such.

Well, the CGI module hasn't had many changes.  There was this bug fix a few
months back:

http://svn.python.org/view?rev=64447&view=rev

Diff to previous:
http://svn.python.org/view/python/trunk/Lib/cgi.py?rev=64447&r1=58218&r2=64447

For all revisions:
http://svn.python.org/view/python/trunk/Lib/cgi.py?rev=67528&view=log

It is possible that by fixing a bug, they brought the behavior in line with
what it *should* be.  Or maybe the browser behavior changed?  The server
does not care about an "action" attribute.  That only tells the browser
where to send the data.  It is possible the browser did not properly format
a request when there was no "action" attribute. Can you provide more
details?

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: More mod_wsgi weirdness: process restarts on redirect

2009-01-29 Thread Joshua Kugler
Ron Garret wrote:
> My question is: is this supposed to be happening?  Or is this an
> indication that something is wrong, and if so, what?

You are probably just hitting a different instance of Apache, thus the
different process ID.

j

--
http://mail.python.org/mailman/listinfo/python-list


Disputing Cookie.py's definition of valid characters

2008-12-11 Thread Joshua Kugler
Some investigation today revealed that Cookie.py thinks these are valid
characters for names and values of cookies:

_LegalCharsPatt  = r"[\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=]"

The rest, presumably being encoded via %NN.

I notice that (), {}, and others made the list, but not [].

Is there a reason for that?  I didn't find a list of valid characters
anywhere in the cookie RFC (disclaimer: I didn't read the HTTP 1.1 RFC).

The reason this got brought up:

I'm parsing some cookies that have been set by a PHP app (out of my control,
no flames please) and it has cookies of the form name[0], name [1], and so
on which get transformed to arrays on the server side such that:

$_COOKIES['name']

Will be a list with values from name[0], name[1].

Well, when these are parsed by Cookie.py, they simply vanish because it
doesn't like the [ and ] in the cookie names.  But these are set
successfully and the browser side, and are sent back to the server
successfully.

Since this is a CherryPy app, it's not as easy as just subclassing the
cookie class I want.  I suppose I could sub class and then monkey patch
CherryPy, but that feels nasty.  Hmm...local copy of Cookie.py that comes
before /usr/lib/python2.5 in the site path would work too.

Thoughts? Is this a bug in Cookie.py, or is PHP out of spec to send cookies
with [ and ] in the name?

j


--
http://mail.python.org/mailman/listinfo/python-list


Re: Possible read()/readline() bug?

2008-10-23 Thread Joshua Kugler
Mike Kent wrote:

> To followup on this:
> 
> Terry: Yes, I did in fact miss the 'buffer' parameter to open.
> Setting the buffer parameter to 0 did in fact fix the test code that I
> gave above, but oddly, did not fix my actual production code; it
> continues to get the data as first read, rather than what is currently
> on the disk.  I'm still investigating why.

What OS is your test code one? What OS is your production code on?  As
mentioned read{line} will mirror the OS's underlying stdio.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: fcgi.py on windows?

2008-10-20 Thread Joshua Kugler
bryan rasmussen wrote:

> As per the subject, anyone know of a version of fcgi.py out there
> somewhere that works on windows yet.

They might have ported a version for Python 2.6.  Versions <= 2.5 didn't
have a socket.fromfd() on Windows, so FCGI and SCGI wouldn't work.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: PYTHON WORKING WITH PERL ??

2008-10-16 Thread Joshua Kugler
Pat wrote:
>> Rewrite everything in python.  Save yourself now...while you still
>> can.
>> 
>> ~Sean
> 
> Trust me. Sean is absolutely correct. I'm currently in the process of
> converting a large Perl project to Python (and learning Python at the
> same time) and the improvement in code is incredible.  After you learn
> Python, you'll come to despise Perl.

I'm not a Python fan-boy, but I'm going to have to agree with Sean too.  I
had been using Perl on-and-off for some 10 years, when I finally had to
write a 500-ish line script in Perl for some data processing.  It got messy
in a hurry.  Is this a hash? Is it a reference to a hash? Is it a reference
to a hash of array references?  Gaaah!  I rewrote it in Python, and it was
so much easier to keep all my data structures straight.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Access to Google Calendar

2008-10-09 Thread Joshua Kugler
pepitovadecurt wrote:
> Hi I need to access to the Google Calendar under python.
> Is posible?

You mean this? 

http://code.google.com/apis/calendar/developers_guide_python.html

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Python - ubuntu

2008-08-29 Thread Joshua Kugler
> A friend of mine told me something about Guido and google developing
> an Ubuntu distribution based and totaly oriented for the Python
> appliction development. I googled for it with no results. Is it
> possible that My Buddy is trying to foole me or is it possible that
> someone knows something about that.

In addition to Fredrick's reply: Canonical, the company that develops
Ubuntu, uses Python *extensively* for their development, so when it comes
to the number of Python modules assembled as packages for a package manager
(apt, in the case of Ubuntu), Ubuntu will probably have the most.  I use
Ubuntu for Python development, and find it very well suited.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: SimpleJson is slow .... is there any C Compiled version ?

2008-07-28 Thread Joshua Kugler
sanket wrote:

> Hello All,
> 
> I have created an API which fetches some data from the database.
> I am using simplejson to encode it and return it back.
> 
> Now the problem is that, this API is being called for millions of
> times in a sequence.
> I ran a profiler and saw that most of the time is consumed in encoding
> my database results in to json.
> So I was just wondering is there any C compiled version of simplejson
> is available?
> or any help regarding this issue would be appreciated.

simplejson does have a C module.  It is compiled automatically when
installed.  If you're installing on Windows, then it probably isn't getting
compiled.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Framework recommendations for web service?

2008-07-16 Thread Joshua Kugler
Phillip B Oldham wrote:
> So, can anyone suggest a lightweight python framework which just does
> the essentials?

web.py is pretty slim (not to be confused with web2py).

Pylons isn't very large, depending on what you call "essential."

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Is re.findall guaranteed to be "in order?"

2008-07-16 Thread Joshua Kugler
Fredrik Lundh wrote:

> Joshua Kugler wrote:
> 
>> Experimenting has shown me that re.findall() will return a list with the
>> matches in the order it found them.
> 
> "in the order it found them" doesn't really say much, does it? ;-)
> 
> "findall" and "finditer" both scans the string from left to right, and
> will return matches in that order.
> 
> (the documentation is a bit vague, so if you have the time, please add a
> request for clarification to the bug tracker)

Done.

http://bugs.python.org/issue3384

j


--
http://mail.python.org/mailman/listinfo/python-list


Is re.findall guaranteed to be "in order?"

2008-07-15 Thread Joshua Kugler
I tried looking through re.py and sre_compile.py, but I can't really see an
answer to my question.

Experimenting has shown me that re.findall() will return a list with the
matches in the order it found them.  Am I lucky so far, or does the
_sre.c's logic dictate the items will be added in order found?

http://svn.python.org/view/python/tags/r25c1/Modules/_sre.c?rev=51334&view=auto
(search for "pattern_findall") seems to indicate they will be added in
order, but can someone with a better handle on Python internals (and quite
frankly, C) confirm this?

Thanks!

j


--
http://mail.python.org/mailman/listinfo/python-list


Re: Problems Returning an HTTP 200 Ok Message

2008-07-10 Thread Joshua Kugler
Guy Davidson wrote:

> Hi Folks,
> 
> I'm having some issues with an small socket based server I'm writing,
> and I was hoping I could get some help.
> 
> My code (attached below) us supposed to read an HTTP Post message
> coming from a power meter, parse it, and return a proper HTTP 200 Ok
> message. The problem is that the socket fails to send the entire
> message as one message, creating a fragmented message which the power
> meter then fails to read and accept.
> 
> Is there any way to force the socket to send the entire message at
> once? Am I doing anything wrong? Is there an easier way to implement
> this functionality?

I don't have a solution to the problem presented, per se, but you might have
an easier time implementing this if you use SimpleHTTPServer or
CGIHTTPServer modules.  This will take care of all the socket listening,
and a lot of other low-level details.

Hope that helps!

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Classes for processing/parsing Wiki Markup text

2008-07-01 Thread Joshua Kugler
Michael Mabin wrote:

> Does anyone know if there are any generally available classes for parsing
> various wiki markup formats?

Several here:

http://pypi.python.org/pypi?%3Aaction=search&term=wiki&submit=search

I'm sure you could find a good one, and add your own markup if needed.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: I Need A Placeholder

2008-06-26 Thread Joshua Kugler
Ampedesign wrote:

> I'm trying to build a try/except case, and I want to have the except
> function like such:
> 
> try:
>   # Do some code here
>   var = 1 # For example
> except:
>   #Do nothing here
> 
> The only problem is if I leave a comment only in the except block, I
> get an error back saying that the except block is not properly
> formatted, since it has no content other than a comment.
> 
> So if anyone could suggest some code to put there as a placeholder
> that would be wonderful.

except:
pass

is the usual technique there.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Pylons and memory use?

2008-06-05 Thread Joshua Kugler
John Salerno wrote:
> Hi everyone. I was thinking about signing up with a web host that
> supports Pylons (among many other things) and one of the differences
> between the various plans is application memory for long-running
> processes. The plan I'd like to sign up for has 80MB. Does anyone know
> if this is enough for basic Pylons applications?
> 
> Just in general, how exactly can I calculate how much memory a Pylons
> application (or any other type of application, for that matter) will
> require? Is there some general range I might be able to rely on? Does
> 80MB seem like enough for just playing around and hobbyist work?

It would depend on what they define as "long running processes."  If the
Python setup is correct, I would assume that it would be running under
mod_python or FastCGI or some such, and would not be considered a "long
running process" as such.  I would assume the long running process would be
something other than the web app you want running in the back ground. Maybe
a queue processor, or something along those lines.

You may wish to ask for clarification.  Sorry I can't help regarding memory
usage, you may wish to ask in the Pylons IRC channel or on their mailing
list.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Proof that \ is a better line joiner than parenthetical sets

2008-06-05 Thread Joshua Kugler
John Salerno wrote:
> According to the Zen of Python, "explicit is better than implicit", and
> the section in the Reference Manual describing the \ line joiner is called
> "Explicit line joining" and the section describing parentheticals is
> called "Implicit line joining."
> 
> So there! ;)

However, according to the Zen of Python, the line before 

"Explicit is better than implicit."

says:

"Beautiful is better than ugly."

And I think putting parenthesis around a multi-line statement is much
prettier.

So there! :)

j


--
http://mail.python.org/mailman/listinfo/python-list


Re: How to covert ASCII to integer in Python?

2008-05-30 Thread Joshua Kugler
Skonieczny, Chris wrote:

> YOU SHOULD REMOVE or CORRECT YOUR POST here:
> http://mail.python.org/pipermail/python-list/2007-February/427841.html
>  
> It is not true - eg. try :
> a='P'# P is ASCII , isn't it ?
> b=int(a)
> and what you will get ? An error !!!
>  
> Or probably you yourself should - quote :
> "You probably should go through the tutorial ASAP that is located here:
> 
> http://docs.python.org/tut/ "

int() converts a strings that is a valid intenter.  What you're looking for
is ord().

In [1]: ord('d')
Out[1]: 100

In [2]: a='P'

In [3]: b=ord(a)

In [4]: b
Out[4]: 80

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: IDE for Python

2008-05-16 Thread Joshua Kugler
Jonathan Barbero wrote:
>  I´m newbie with Python and to learn it better I want to use a good IDE to
> concentrate on Python power. There is any IDE like Eclipse for Java for
> Python? If not, what is the best Python´s IDE for you?

This is an EFAQ (extremely frequently asked question).

http://www.google.com/search?hl=en&q=python+ide+site%3Amail.python.org%2Fpipermail%2Fpython-list%2F&btnG=Google+Search

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Saving an audio file's waveform into an image

2008-05-12 Thread Joshua Kugler
Julien wrote:

> Hi,
> 
> I would like to pull out the waveform of an audio file and save it
> into an image file, for example in GIF format. Is that achievable, and
> if so, how?

Take a look at http://code.enthought.com/projects/chaco/  One of their
examples does exactly this. Last example on this page:
http://code.enthought.com/projects/chaco/gallery.php

Hope that helps.

j


--
http://mail.python.org/mailman/listinfo/python-list


Re: Setting an attribute without calling __setattr__()

2008-04-29 Thread Joshua Kugler
animalMutha wrote:

>> Consider reading the *second* paragraph about __setattr__ in section
>> 3.4.2 of the Python Reference Manual.
> 
> if you are simply going to answer rtfm - might as well kept it to
> yourself.

For what it's worth, I (the original poster) am glad he answered that way. 
It showed me the section and paragraph I had overlooked when reading
through the docs the first time.

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Setting an attribute without calling __setattr__()

2008-04-25 Thread Joshua Kugler
John Machin wrote:
>> Is there a way to define self.me without it firing __setattr__?
> Consider reading the *second* paragraph about __setattr__ in section
> 3.4.2 of the Python Reference Manual.

Like I said in my original post, it was probably staring me right in the
face.  I had read through a bit of the documentation on special methods,
but for some reason I missed that part.

Thanks to all for your responses!

j



--
http://mail.python.org/mailman/listinfo/python-list


Setting an attribute without calling __setattr__()

2008-04-25 Thread Joshua Kugler
OK, I'm sure the answer is staring me right in the face--whether that answer
be "you can't do that" or "here's the really easy way--but I am stuck.  I'm
writing an object to proxy both lists (subscriptable iterables, really) and
dicts.

My init lookslike this:

def __init__(self, obj=None):
if type(obj).__name__ in 'list|tuple|set|frozenset':
self.me = []
for v in obj:
self.me.append(ObjectProxy(v))
elif type(obj) == dict:
self.me = {}
for k,v in obj.items():
self.me[k] = ObjectProxy(v)

and I have a __setattr__ defined like so:

def __setattr__(self, name, value):
self.me[name] = ObjectProxy(value)

You can probably see the problem.

While doing an init, self.me = {} or self.me = [] calls __setattr__, which
then ends up in an infinite loop, and even it it succeeded

self.me['me'] = {}

is not what I wanted in the first place.

Is there a way to define self.me without it firing __setattr__?

If not, it's not a huge deal, as having this class read-only for now won't
be a problem, but I was just trying to make it read/write.

Thanks!

j

--
http://mail.python.org/mailman/listinfo/python-list


Re: Google App Engine

2008-04-09 Thread Joshua Kugler
Duncan Booth wrote:
>
http://www.techcrunch.com/2008/04/07/google-jumps-head-first-into-web-services-with-google-app-engine/
> "Python only. What a weird decision. Not business and
> community-friendly at all."

Translation: "Not
over-engineered-top-heavy-overly-complex-job-security-inducing-Java-enterprise
friendly."

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Can C.L.P.handle the load?

2008-04-09 Thread Joshua Kugler
Paddy wrote:
> I'm waiting for the rush of new users to c.l.p :-)
> If it comes, then aren't regularly posted FAQ's newbie friendly?

No, it just means they didn't take the time to read the docs and the FAQ for
themselves. :)

> Is their a good FAQ already around that we can regularly point newbies
> to?

You mean this one: http://www.python.org/doc/faq/

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Easy-to-install-and-use scientific graphics (plotting) package for Python 2.5.1 on Linux Ubuntu 7.1

2008-04-04 Thread Joshua Kugler
> Is there an easy scientific graphics (plotting) package for Python
> 2.5.1 running on Ubuntu Linux 7.1 ("Gutsy Gibbon")?
> 
> A few years ago I used PyLab (a MatLab-like plotting module for
> Python) on a Windows machine, but I don't know if there is a similar
> easy-to-install-and-use Python 2.5.1-compatible graphics package for
> Ubuntu Linux 7.1?

We've been using Matplotlib for a while, but are looking for alternatives. 
I recently came across Chaco (http://code.enthought.com/chaco/) and am
quite impressed with what I see so far.

It's API is very clean, not requiring the "magic functions" (can we say
setp()?) that is required by matplotlib.  Very OO, and from what I see of
example code, the API complies with PEP 8 conventions.

Installation was easy (easy_install), but for a couple packages, you will
need GCC, python-dev, and swig.

I haven't dug into it much, but it looks really, really promising.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: SoC project: Python-Haskell bridge - request for feedback

2008-03-27 Thread Joshua Kugler
Micha? Janeczek wrote:

> Hi,
> 
> I am a student interested in participating in this year's SoC.
> At http://tsk.ch.uj.edu.pl/~janeczek/socapp.html (and also below
> in this email) you can find a draft of my project proposal.
> 
> I'd like to ask you to comment on it, especially the deliverables
> part. Are you interested in such a project, and if yes, what features
> would be most important to you? Is anything missing, or should
> something get more priority or attention?

You might want to take a look at this: 

http://www.artfulcode.net/articles/extending-python-almost-anything/

That might get you started at least on the calling Haskell part.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Not understanding lamdas and scoping

2008-03-26 Thread Joshua Kugler
George Sakkis wrote:

> On Mar 26, 5:02 pm, Joshua Kugler <[EMAIL PROTECTED]> wrote:
> 
>> I am trying to use lamdba to generate some functions, and it is not
>> working
>> the way I'd expect.  The code is below, followed by the results I'm
>> getting.  More comments below that.
>>
>> (...)
>>
>> So, is there some scoping issue with lambda
>> that I'm not seeing?
> 
> Yes; it's not related to lambda though but to closures (whether
> defined as lambdas or regular functions). See for example
>
http://groups.google.com/group/comp.lang.python/browse_frm/thread/94d1bba8ad56baf4
> 
> There should be an entry for this at
> http://www.python.org/doc/faq/programming/, that's really an FAQ.

George -

Thanks for the quick and clear answer...that makes perfect sense.  To
bad...I'd like to just use a list comprehension. :)

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Not understanding lamdas and scoping

2008-03-26 Thread Joshua Kugler
I am trying to use lamdba to generate some functions, and it is not working
the way I'd expect.  The code is below, followed by the results I'm
getting.  More comments below that.

patterns = (
('[sxz]$', '$','es'),
('[^aeioudgkprt]h$', '$', 'es'),
('[^aeiou]y$', 'y$', 'ies'),
('$', '$', 's'),
)

def z(*a):
print a

def make_pattern(pattern, search, replace):
def compare(word):
z(pattern, search, replace)
return compare

rules = [make_pattern(pattern, search, replace)
 for (pattern, search, replace) in patterns]

print 'make pattern'
for rule in rules:
rule('noun')

rules = [lambda word: z(pattern, search, replace)
for (pattern, search, replace) in patterns]

print 'lambda/list comprehension'
for rule in rules:
rule('noun')

rules = []
for pattern, search, replace in patterns:
rules.append(lambda word: z(pattern, search, replace))

print 'lamda/for loop'
for rule in rules:
rule('noun')

Ouptut:

make pattern
('[sxz]$', '$', 'es')
('[^aeioudgkprt]h$', '$', 'es')
('[^aeiou]y$', 'y$', 'ies')
('$', '$', 's')
lambda/list comprehension
('$', '$', 's')
('$', '$', 's')
('$', '$', 's')
('$', '$', 's')
lamda/for loop
('$', '$', 's')
('$', '$', 's')
('$', '$', 's')
('$', '$', 's')

Of course, in my real code, I'm not calling z(), but a couple of RE
search/replace functions (yes, I'm working on the Dive Into Python
pluralizer).

The first result is obviously the correct one.  But the next two don't make
sense.  Why do all the functions returned by lamda have the last input
values?

What am I missing?

OK, I figured out if I did this:

def make_pattern(pattern, search, replace):
return lambda word: z(pattern, search, replace)

it would give correct results.  So, is there some scoping issue with lambda
that I'm not seeing?

Thanks!

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Comunicate processes with python

2008-03-18 Thread Joshua Kugler
Adrián Bravo Navarro wrote:
>> Is there any simple way to achieve this goal? We've been thinking of
>> sockets but Im not conviced at all with that.

If you want to communicate between processes on the same host, yes, you can
use DBus or a couple of the options here:
http://docs.python.org/lib/ipc.html

If you want to communicate between hosts, then sockets is probably going to
be your only options, although there are libraries that abstract some of
that for you to make it easier to manage.  You might want to take a look at
Pyro. http://pyro.sourceforge.net/

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How to import custom python file in python server page (psp) ?

2008-03-14 Thread Joshua Kugler
James Yu wrote:

> Hi folks,
> 
> I prepared a python script for dynamically get the absolute paths of the
> files in certain folder.
> Then I tried to invoke that function from my web server in a .psp file
> like this:
> 
>   1 
>   2 
>   3 asdfasdfasdfa
>   4 
>   5 <%
>   6 import glob
>   7 import os
>   8 *import Helper
> *  9
>  10 body = ''
>  11 top = 'asdfasdfasdfa'
>  12 links = {}
>  13 *Helper.GetLinks(top=top)
> * 14 *paths = Helper.GenLinkPath(links)
> * 15 body = paths
>  16 %>
>  17 <%=body%>
>  18 
>  19 
> 
> However, this is the error message I received when I open the page in a
> browser:
> 
>> Mod_python error: "PythonHandler mod_python.psp"
>>
>> Traceback (most recent call last):
>>
>>   File "/usr/lib/python2.5/site-packages/mod_python/apache.py", line 299,
>> in HandlerDispatch
>> result = object(req)
>>
>>   File "/usr/lib/python2.5/site-packages/mod_python/psp.py", line 302, in
>> handler
>> p.run()
>>
>>   File "/usr/lib/python2.5/site-packages/mod_python/psp.py", line 213, in
>> run
>> exec code in global_scope
>>
>>   File "/var/www/.cyu021/.pic/index.psp", line 8, in
>> import Helper
>>
>> ImportError: No module named Helper
> 
> 
> *PS. I put Helper.py and index.psp in the same dir
> *
> Thanks in advance,


What is the import path?  The current directory in PSP might not be the
directory in which the .psp file resides.  Print out sys.path before you
import your helper module to see what paths you're dealing with.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: problem with mod_python

2008-02-19 Thread Joshua Kugler
Pradnyesh Sawant wrote:

> Hello,
> I have a small program which does 'import hashlib'. This program runs fine
> with python2.5. But when I try running the same program through
> mod_python, I get the error: 'ImportError: No module named hashlib' in the
> apache2 error.log
> 
> Searching online suggested me to include md5.so or md5module.so in
> apache2. but I don't see that in a package for debian lenny (the system
> I'm using).
> 
> So, my Q is, is it possible to make mod_python use the same PYTHONPATH as
> the python2.5 interpreter? if so, how?

It sounds like your mod_python may be compiled against a different version
of Python than your main installation?  How did you install mod_python? How
did you install your main python installation?

What is the output of the command:

ldd /path/to/mod_python.so

(the full path on my system is /usr/lib/apache2/mod_python.so)

There should be a line something like:

libpython2.5.so.1.0 => /usr/lib/libpython2.5.so.1.0 (0xb7e37000)

If it is pointing to libpython.2.4.so.1.0, then that could be the reason for
you troubles.

Hope that helps.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: No Module Named pstats

2008-02-13 Thread Joshua Kugler
Chris wrote:

> On Feb 13, 11:20 am, "Juha S." <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> I'm trying to use the Python profilers to test my code, but I get the
>> following output for cProfile.run() at the interpreter:
>>
>> Traceback (most recent call last):
>>   File "", line 1, in 
>>   File "/usr/lib/python2.5/cProfile.py", line 36, in run
>> result = prof.print_stats(sort)
>>   File "/usr/lib/python2.5/cProfile.py", line 80, in print_stats
>> import pstats
>> ImportError: No module named pstats
>>
>> I also tried to use the profile module but I get:
>>
>>  >>> import profile
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> ImportError: No module named profile
>>
>> I'm on Ubuntu 7.10 with Python 2.5, and I can't seem to figure out
>> what's missing.
> 
> There is no module called 'profile', it's called 'cProfile'.  Both
> 'cProfile' and 'pstats' are part of the standard distro, have you
> tried downloading a new binary and installing it ?

There is a module called profile, as well as cProfile.  And as Ilias said,
you probably do have to install python-profile (or whatever package your
system places the profiler in).

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: embedded python in c++ packaging

2008-02-07 Thread Joshua Kugler
Furkan Kuru wrote:
> Hello,
> 
> I have been developing an application in C++ that embeds Python
> interpreter. It takes advantage of too many modules from Python.
> When I want to package this application, I need to add too many files
> (.pyc) from Python/lib folder together with Python25.dll.
> Is there a way to pack these .pyc files to a zip file and redirect
> Python25.dll to that zip file?

That is effectively what py2exe does with the modules required by the main
application.  It takes all the required modules and puts them in a
library.zip file.  You might take a look at how it does it.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: SimpleXMLRPCServer to fastcgi via WSGI?

2008-02-01 Thread Joshua Kugler
Ivan Voras wrote:
> Is there a straightforward way to convert an XML-RPC server application
> (written for SimpleXMLRPCServer) to use WSGI so that it can be used as s
> fastcgi server? By "straightforward" I mean something simple, without
> using some external framework.
> 
> Alternatively, I don't really care about WSGI, so something that makes
> it work as fastcgi directly will also be ok.

You don't need WSGI to use FastCGI.  See
http://pypi.python.org/pypi/python-fastcgi  It says it includes WSGI server
implementations, but that doesn't mean you have to use them.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Trying to understand Python web-development

2008-01-29 Thread Joshua Kugler
walterbyrd wrote:
> Python also seems to require some sort of "long running processes" I
> guess that the python interpretor has to running all of time.

What you probably don't realize, is that in 99.9% of the situations you've
come across, PHP is already a process running all the time.  It's called
mod_php, and it's a PHP interpreter running inside of apache, thus a "long
running process."  It's just not as obvious.  You can do the same thing
with Python via the mod_python module, thus putting the python interpreter
(instead of the PHP interpreter) inside the Apache process. Other web
servers have similar setups.  Google for FastCGI and SCGI.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python and binary compatibility

2008-01-25 Thread Joshua Kugler
Christian Heimes wrote:
> You can use MinGW32 to compile the extension, too. Or use the free
> toolchain as described at
> http://wiki.python.org/moin/Building_Python_with_the_free_MS_C_Toolkit

That page has a link to the Microsoft Visual C++ Toolkit 2003 page, which
then says it's been discontinued and to use Visual C++ 2005 Express
Edition. Sigh...

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Perl Template Toolkit: Now in spicy new Python flavor

2008-01-15 Thread Joshua Kugler
[EMAIL PROTECTED] wrote:
> I'd like to inform the Python community that the powerful and popular
> Template Toolkit system, previously available only in its original
> Perl implementation, is now also available in a beta Python
> implementation:
> 
> http://tt2.org/python/index.html
> 
> I created this port both as a fun programming project, and for use in
> environments where  Perl is not available, for reasons technical,
> cultural, or otherwise.  The extensive Perl test suites have also been
> ported, and most templates require no or very little modification.

I must say...wow.  That would have saved me some time and hassle about a
year ago, but of course, I wouldn't have fell in love with pure XML
templates either. :)  As someone who has used Template Toolkit quite a bit,
I must say that is is quite cool.  Congrats on a job well done!

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: split parameter line with quotes

2008-01-11 Thread Joshua Kugler
teddyber wrote:
> first i'm a newbie to python (but i searched the Internet i swear).
> i'm looking for some way to split up a string into a list of pairs
> 'key=value'. This code should be able to handle this particular
> example string :
> 
> qop="auth,auth-int,auth-conf",cipher="rc4-40,rc4-56,rc4,des,
> 3des",maxbuf=1024,charset=utf-8,algorithm=md5-sess
> 
> i know i can do that with some regexp (i'm currently trying to learn
> that) but if there's some other way...

Take a look at the shlex module.  You might be able to fiddle with the shlex
object and convince it to split on the commas.  But, to be honest, that
above would be a lot easier to parse if the dividing commas were spaces
instead.

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: .NET and Python Integration Problem and PDF Library (Need Help and Suggestions)

2007-12-18 Thread Joshua Kugler
Ravi Kumar wrote:
> - your opinion with available PDF Libraries, that are best among. Also
> which library to use for Windows server platform (there is limitation
> on installing long chain libraries that include other deep
> dependencies too). A pure python PDF library would be good, but which
> one.
> -Which XML Library is pure python based.

Never done Python/.NET integration, so I can't help you there.

PDF library: ReportLab. But that is most generation of PDFs.  For reading,
splitting, etc, you may have to look at their commercial offering.  And I
do believe it is all pure python.

XML: ElementTree  http://effbot.org/zone/element-index.htm  Should be all
you need.

Hope that helps some.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Handling cookies without urllib2 and cookielib

2007-12-17 Thread Joshua Kugler
Gabriel Genellina wrote:

> On 14 dic, 23:44, Joshua Kugler <[EMAIL PROTECTED]> wrote:
> 
>> I'm using HTTPlib to construct some functional tests for a web app we're
>> writing.  We're not using urllib2 because we need support for PUT and
>> DELETE methods, which urllib2 does not do.
>>
>> We also need client-side cookie handling.  So, I start reading about
>> cookielib and run across a problem.  It's cookie handling is tied quite
>> tightly to urllib2's request object.  httplib has somewhat different
>> semantics in its request object.  So, you can use cookielib with httplib.
>> And cookie lib has no simple function (that I could find) for passing in
>> a set-cookie header and getting back a CookieJar object (or even a list
>> of Cookie objects).
> 
> What about correcting the first thing, making urllib2 support HEAD/PUT/
> DELETE?


We may have to do that, and then hack on the Redirect handler too so it will
properly keep the request method.  But that's not our preference, for
obvious reasons. :)

I just find it hard to believe that no one has ever needed to do cookie
handling in a generic way (i.e. input: set-cookie header, output: cookie
objects) before.  May have to write my own. Or sublcass/extend cookielib.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: why this error?

2007-12-16 Thread Joshua Kugler
python.jiang wrote:

> hello friends, the question had show bellow, any friend can tell me why.
> thanks.
>  
> list:
> def test():
>   exec "import sys"
>   a=range(15)
>   b=[13,3]
>   c=filter(lambda x: x not in b,a)
>   return c
> print test()
>  
> run result:
>   File "a.py", line 2
> exec "import sys"
> SyntaxError: unqualified exec is not allowed in function 'test' it
> contains a nested function with free variables

I have no reason why you're getting the error, but there is no reason to try
to exec an import.  Just use

import sys

Hope that helps.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Handling cookies without urllib2 and cookielib

2007-12-14 Thread Joshua Kugler
Standard disclaimer: read, googled, read some more.  If you have a link,
please free free to point me there.

I'm using HTTPlib to construct some functional tests for a web app we're
writing.  We're not using urllib2 because we need support for PUT and
DELETE methods, which urllib2 does not do.

We also need client-side cookie handling.  So, I start reading about
cookielib and run across a problem.  It's cookie handling is tied quite
tightly to urllib2's request object.  httplib has somewhat different
semantics in its request object.  So, you can use cookielib with httplib. 
And cookie lib has no simple function (that I could find) for passing in a
set-cookie header and getting back a CookieJar object (or even a list of
Cookie objects).

I'm sure I'm not the first to have to deal with httplib and cookies.  Anyone
have suggestions or pointers?

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: reading cookies from PHP

2007-12-10 Thread Joshua Kugler
Jack Holt wrote:
> Hello there.
> I'm a PHP fan but a Python newbie. I wrote anapplication in Python
> that needs to read a cookie setup from a PHP page. Is itpossible to do it?
> 
> If not, what if I create a TXT file - as well as a cookie - thatcontains
> the cookie's data? Will python be able to open the file and readthe
> data? The TXT file can contain:
> 
> Username = dude
> Password = python

Cookies are sent browser by the web server, and sent back to the web server
by the browser.  So, yes, you can set the cookie with the PHP script and
read it with the python program.

I do it like this:

import Cookie
myCookies = Cookie.SimpleCookie()

if 'HTTP_COOKIE' in os.environ:
myCookies.load(os.environ['HTTP_COOKIE'])

After that, the cookies will be available via a dictionary style lookup:

user_name = myCookies['user_name']

j



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Capture DOS error???

2007-12-06 Thread Joshua Kugler
Tangen, Erik wrote:

> Did you ever get a solution to this?
> I am also in need of this solution.
> Thanks,
> Erik

Can you clarify?  What is a DOS error?  You mean an error in reading or
writing a file?  That would be an IOError exception.  If a file is not
found, or you can't read it, that's an OSError exception.  Beyond that, I'm
not sure what you're asking for.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: JSON

2007-12-06 Thread Joshua Kugler
[EMAIL PROTECTED] wrote:

> i tried a couple python json libraries. i used simplejson on the
> server and was using cjson on the client, but i ran into this issue.
> i'm now using simplejson on both sides, but i'm still interested in
> this issue.  did i do something wrong? is there a bug in one of the
> libraries? or something i don't understand about the json spec?
> 
> 
> i problem is the line where i call cjson.decode() below:
> 
 import simplejson
 import cjson

 sj = simplejson.dumps('http://server.com')
 sj
> '"http:\\/\\/server.com"'
 cj = cjson.encode('http://server.com')
 cj
> '"http://server.com";'
 simplejson.loads(cj)
> u'http://server.com'
 cjson.decode(cj)
> 'http://server.com'
 simplejson.loads(sj)
> u'http://server.com'
 cjson.decode(sj)
> 'http:\\/\\/server.com' # is this correct

Known issue.  See:
http://blog.extracheese.org/2007/07/when-json-isnt-json.html

Neither project has fixed it it seems.  Not sure which is actually
the "correct" way to do it, but it would be nice if they would agree.

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: which configparse?

2007-12-06 Thread Joshua Kugler
Shane Geiger wrote:

> Best, is naturally, a somewhat subjective evaluation.  That being said,
> configparser is well regarded.  I have also seen these two options that
> you might want to check out:
> 
> http://wiki.woodpecker.org.cn/moin/Dict4Ini
> http://www.voidspace.org.uk/python/configobj.html

+1 on configobj.  The ability to have mutli-level config files is wonderful,
and it's all dict and/or array access methods.  Wonderful.

>> I'd like to make minimal change to my working optparse setup (there are
>> lots of options - I don't want to duplicate all the cmdline parsing with
>> ini file parsing code)

I found (at least) two areas where lots of code is unavoidable:
configuration handling and error handling.  That said, you could use
configparse (or configobj) to set the defaults of all your command line
options, like so:

cfg = configobj.ConfigObj('file.ini', file_error=True)
parser.add_option('--opt', dest='opt',  default=cfg.get('opt', None)

Hmm...now why didn't I do that in my code?  Thanks for the idea! :)

j


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: (More) Re: Is __import__ known to be slow in windows?

2007-12-01 Thread Joshua Kugler
John Machin wrote:

> On Dec 1, 2:12 pm, Joshua Kugler <[EMAIL PROTECTED]> wrote:
>> x = __import__(m)
> 
> Have you ever tried print m, x.__file__ here to check that the modules
> are being found where you expect them to be found?

No, I haven't, but I do know for a fact that the only location of the module
found is where I think it is.  There are no other modules on the system (or
in the search path) named _modulename.

>> except ImportError, e:
>> if not e.message.startswith('No module named'):
>> raise
> 
> Why are you blindly ignoring the possibly that the module is not
> found? Note that loading a module after it is found is not a zero-cost
> operation.

That was in the original code to silently ignore things like a README file
or a directory that didn't have a __init__.py file.  The code I posted was
from my benchmarking script.  The code in the framework does a listdir()
and goes through that list trying to import.  We'll probably "smarten up"
that code a bit, but for now it works.

>> x = None
>>
>> Each of those three module names is a directory under /path/to/code with
>> an empty __init_.py.
> Is there anything else in the /path/to/code directory?

A directory and a file that aren't referenced in my test, but are
in the framework's import attempts (see listdir() comment above). 
Regardless, setting sys.path to just one directory speeds up both the
benchmark and the framework.

>> def r():
>> for m in ['three','module','names']:
>> x = __import__(m)
> 
> Have you tried print m, x.__file__ here to check that the modules are
> being found where you expect them to be found?

As I said above, no I haven't, but I will just to double check.

On Linux, print x, x.__name__ produces the expected result whether I have
one element in sys.path or all of them. On Windows, same result.

> Call me crazy, but: First experiment, sys.path was ['/path/to/code',
> '', etc etc]. Now it's only ['/path/to/code']. How can that still load
> properly but run faster??

I don't know, but it does.  Right now, we're just importing empty modules. 
Once those modules have imports of their own, my "fix" will probably fall
apart, and we'll be back to square one.

> What directory tree walking?? Should be none 
> if the modules are found in /path/to/code.

I know, that's the crazy thing...there should be no directory tree walking
or other filesystem activity after finding the module.  But setting
sys.path to one element does make a difference; that can be seen.  I don't
know the internal implementation of __import__, so I can't really comment
beyond my results.  Maybe I need to strace a python run.

> Are you sure the modules 
> are always found in /path/to/code? What is in the current directory
> [matched by '' in sys.path]?

'' would be whereever I start the benchmark script (~/bin), or whereever the
framework is started from, probably from the web server's home directory,
or from ~/bin when I'm profiling the framework.

Thanks for your responses.  I still trying to figure this out too. :)

j

-- 
http://mail.python.org/mailman/listinfo/python-list


(More) Re: Is __import__ known to be slow in windows?

2007-11-30 Thread Joshua Kugler
Ok, so we have this code:

t = timeit.Timer(stmt='r()', setup='from __main__ import r')

sys.path.insert(0,'/path/to/code')

def r():
for m in ['three','module','names']:
try:
x = __import__(m)
except ImportError, e:
if not e.message.startswith('No module named'):
raise
x = None

Each of those three module names is a directory under /path/to/code with an
empty __init_.py.

On Linux, the run of that code for 1000 iterations takes 0.014 CPU seconds
with 0.007 of that spent in __import__().

On Windows (with on-access checking turned off), 1000 iterations takes 7.9
seconds, with 7.851 seconds of that spent in __import__().

Let's try something...let's modify the code a bit:

sys.path = ['/path/to/code']

def r():
for m in ['three','module','names']:
x = __import__(m)
x = None

cProfile.run('t.timeit(number=%s)' % number, 'stat_file')
p = pstats.Stats('stat_file')
p.sort_stats('time', 'cum').print_stats(.5)

Now, with only my directory in sys.path, the run times are:
Linux: 0.0013 (0.006 spent in __import__)
Windows: 0.0012 (0.007 spent in __import__)

So, it appears walking the directory trees is what is costing the time.

sys.path on Windows:
['', 'C:\\Python25\\lib\\site-packages\\paste-1.5.1-py2.5.egg',
'C:\\WINDOWS\\system32\\python25.zip',
'C:\\Python25\\DLLs',
'C:\\Python25\\lib',
'C:\\Python25\\lib\\plat-win',
'C:\\Python25\\lib\\lib-tk',
'C:\\Python25',
'C:\\Python25\\lib\\site-packages',
'C:\\Python25\\lib\\site-packages\\win32',
'C:\\Python25\\lib\\site-packages\\win32\\lib',
'C:\\Python25\\lib\\site-packages\\Pythonwin']

On Linux:
['', '/usr/lib/python2.5/site-packages/setuptools-0.6c7-py2.5.egg',
'/usr/lib/python2.5/site-packages/simplejson-1.7.3-py2.5-linux-i686.egg',
'/usr/lib/python2.5/site-packages/Paste-1.5.1-py2.5.egg',
'/usr/lib/python2.5/site-packages/TracWebAdmin-0.1.2dev-py2.5.egg',
'/home/jkugler/programming',
'/usr/lib/python25.zip',
'/usr/lib/python2.5',
'/usr/lib/python2.5/plat-linux2',
'/usr/lib/python2.5/lib-tk',
'/usr/lib/python2.5/lib-dynload',
'/usr/local/lib/python2.5/site-packages',
'/usr/lib/python2.5/site-packages',
'/var/lib/python-support/python2.5',
'/usr/lib/site-python']

So, not identical (actually, longer on linux) but similar.

Interesting, at any rate.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Is __import__ known to be slow in windows?

2007-11-30 Thread Joshua Kugler
> What modules are you __import__ing, and what is platform-dependent in
> each?

The only thing we're importing __import__ are some modules of ours, with no
sytem dependent code in them at all.  Some of them are even empty modules,
as was suggested by one response (for benchmarking purposes).

Turning off Symantec's on-access checking shaved about a bunch of time off
the time spent in __import__ (10 seconds vs 16), but still too high.

Commenting out the import of the only system dependent code we have in our
project (which we don't "manually" import via an __import__ call) produced
no change in run time.

So, we've found a major culprit.  I'll see if I can find more.

Thanks for the pointers so far.

j

-- 
http://mail.python.org/mailman/listinfo/python-list


Is __import__ known to be slow in windows?

2007-11-30 Thread Joshua Kugler
[I tried googling for this, didn't find anything relevant.]

We've recently been doing some profiling on a project of ours.  It runs
quite fast on Linux but *really* bogs down on Windows 2003.  We initially
thought it was the simplejson libraries (we don't use the C extensions) but
profiling proved otherwise.

We have a function that does some runtime imports via calls to __import__. 
We ran 1000 iterations (we used cProfile) of the application (web app).
There were eight calls to __import__ per iteration, so 8000 calls total.
Identical hardware, by the way.

On Linux (Debian Etch, Python 2.5.1)
Total time was 2.793 CPU seconds, with __import__ using 1.059 seconds of
that.  So, 37% of the time was spent in import.  Not great, but not a show
stopper.

On Windows 2003 (R2, Python 2.5.1)
Total time was 18.532 CPU seconds, with __import__ using 16.330 seconds
(88%) of that.

So, Linux spends 1.734 seconds on non-import activities, and Windows spends
2.202 seconds on non-import activities.  Pretty close.  But 16.3 seconds on
import!?

Is this a known deficiency in Windows' Python import calls, or is there
something deeper going on here?

Pointers, suggestions, and URLs welcome.

j



-- 
http://mail.python.org/mailman/listinfo/python-list