Re: Common Python Idioms

[Daniel Dittmar]

Marc 'BlackJack' Rintsch wrote:
> If ``in`` shouldn't work with dictionaries, either `__contains__()` must
> be implemented to throw an exception or dictionaries shouldn't be iterable.

I agree completely (in the sense that dictionaries shouldn't be iterable 
directly). Probably even more strongly, at least every time I see some 
code where someone iterates over the keys, only to use the key to look 
up the value (instead if using iteritms).

For strings, the 1:1 relationship between 'in' and iteration has already 
been broken. But then, iteration over strings happens in my code only 
when a pass a string where I should have passed a list of strings.

Daniel


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Common Python Idioms

[Daniel Dittmar]

John Machin wrote:
> No it doesn't look wrong to anyone who has read the docs on
> sys.modules. 

My point was really that there is no obvious implementation for 'in' on 
dictionaries, so it should have been left out. And that GvR thought so 
for quite some time as well (until he got mixed up with a crowd of ex 
C++ programmers).

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Common Python Idioms

[Daniel Dittmar]

Duncan Booth wrote:
> In this case, dict objects used to not support the 'in' operator, but at 
> some point it was added. I believe it wasn't there originally because Guido 
> wasn't sure whether people would expect it should match keys or keys and 
> values.

And he was right:

import sys
'sys' in sys.modules => True
sys in sys.modules => False

Doesn't this look wrong?

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is python memory shared between theads?

[Daniel Dittmar]

Wesley Henwood wrote:
> So I declare a variable named A in thread1, in script1.py.  I assign
> the value of 2.5 to A.  I then run script2.py in thread2.  Script2.py
> assigns the value of 5.5 to a variable named A.  Now, when thread1
> resums execution, I see that A = 5.5, rather than 2.5 as I expected.
> 
> Is this normal behavior?  

Not if this is all you are doing. A variable A in script1.py and a 
variable A in script2.py are completely different, even when running in 
the same thread.

But if you're running script1.py and script2.py by calling execfile or 
exec and you pass the same dictionary as the globals argument to 
execfile, then the two scripts would share the global namespace. 
Variables of the same name would really be the same variable.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Py3K idea: why not drop the colon?

[Daniel Dittmar]

Michael Hobbs wrote:
> But if the BDFL insists that it remains, why not 
> take the converse approach? That is, assume that the expression ends at 
> the colon, not at the newline. That would make this type of statement 
> possible:

I suggested something like this a while back. The answer then was that 
error messages in case of a forgotten colon would be misleading, at the 
wrong position, or both.

Today:

if a or b
 print x

File "colontest.py", line 2
 if a or b
 ^
SyntaxError: invalid syntax

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exploiting Dual Core's with Py_NewInterpreter's separated GIL ?

[Daniel Dittmar]

robert wrote:
> (IPython is only a special "python network terminal" as already said.)

Sorry, I thought of IronPython, the .NET variant.

> Does Jython really eliminate the GIL? What happens when different 

Yes.

> threads alter/read a dict concurrently - the basic operation in python, 
> which is usually not protected by locks. Does Jython use a dict data 
> type (and other collection types) which lock during _each_ access? in 
> case - that may be very expensive.

True, though Java synchronization has gotten better and better. Still, 
Sun introduced non locking hash tables, so it was a problem. It'd be 
interesting to know which kind of dict is used in Jython for attribute 
access.
> 
> garbage is collected earliest, when the refcount went to 0. If it ever 
> went to 0, no one will ever use such object again. Thus GC should not be 
> different at all.

Since Python 2.?, there's a mark-and-sweep garbage collection in 
addition to the reference counting scheme. This was put into Python to 
be able to reclaim object cycles.


Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exploiting Dual Core's with Py_NewInterpreter's separated GIL ?

[Daniel Dittmar]

robert wrote:
>  Question Besides:  do concurrent INC/DEC machine OP-commands 
> execute atomically on Multi-Cores as they do in Single-Core threads?

No on the level that that Python reference counting is implemented. The 
CPUs have often special assembler ops for these operations. I think that 
it's even more complicated as some machines require additional read and 
write barriers around these operations.

>> Take a look at IPython1 and it's parallel computing capabilities [1,
[...]
> I do selected things with interprocess shared memory techniques already. 
> Thats medium efficent.
> Multiple interpreters inside one process seem to be most promising for 
> seemless multi-core programming. 

Both IPython and Jython use multiple CPUs without the GIL, so you don't 
need to add the complication of multiple interpreters.

 > As all Python objects share the same
 > malloc space - thats the key requirement in order to get the main
 > effect. As soon as with have object pickling in between its well away
 > from this very discussion.

There's the complication that CPython has garbage collection in addition 
to reference counting. I'm not sure what happens when the garbage 
collector looks at objects rooted in another interpreter.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exploiting Dual Core's with Py_NewInterpreter's separated GIL ?

[Daniel Dittmar]

robert wrote:
> I'd like to use multiple CPU cores for selected time consuming Python 
> computations (incl. numpy/scipy) in a frictionless manner.
> 
> Interprocess communication is tedious and out of question, so I thought 
> about simply using a more Python interpreter instances 
> (Py_NewInterpreter) with extra GIL in the same process.

If I understand Python/ceval.c, the GIL is really global, not specific 
to an interpreter instance:
static PyThread_type_lock interpreter_lock = 0; /* This is the GIL */

Daniel


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: can't open word document after string replacements

[Daniel Dittmar]

Antoine De Groote wrote:
> I have a word document containing pictures and text. This documents 
> holds several 'ABCDEF' strings which serve as a placeholder for names. 
> Now I want to replace these occurences with names in a list (members). I 
> open both input and output file in binary mode and do the 
> transformation. However, I can't open the resulting file, Word just 
> telling that there was an error. Does anybody what I am doing wrong?

The Word document format probably contains some length information about 
paragraphs etc. If you change a string to another one of a different 
length, this length information will no longer match the data and the 
document structure will be hosed.

Possible solutions:
1. Use OLE automation (in the python win32 package) to open the file in 
Word and use Word search and replace. Your script could then directly 
print the document, which you probably have to do anyway.

2. Export the template document to RTF. This is a text format and can be 
more easily manipulated with Python.

> for line in doc:

I don't think that what you get here is actually a line of you document. 
Due to the binary nature of the format, it is an arbitrary chunk.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Installing Python on a 64-Bit OS

[Daniel Dittmar]

Nico Grubert wrote:
> I'd like to install Python 2.3.5. on a 64-Bit OS (Suse Linux Enterprise 
> Server 10) on an AMD Opteron 64-Bit machine.
> I have to use Python 2.3.5.
> 
> Do I need a special source archive or can I use "Python-2.3.5.tgz" from 
> http://www.python.org/ftp/python/2.3.5/Python-2.3.5.tgz ?
> 
> Is there anything special I have to care about or is installing Python 
> on a 64 Bit OS just as easy as installing it on a 32-Bit OS?

On this platform, it should be easy to build from the sources as gcc 
builds 64 bit programs by default.

On some platforms (Solaris/Sparc, AIX, HP-UX/PA-RISC), the compiler 
defaults to 32bit programs. You then have to convince configure to use 
the right CFLAGS. The exact technique might differ from configure to 
configure: some require an option, some require an environment variable. 
Tip: create wrapper scripts for the compiler (and possibly ld and ar), 
which uses the proper options. Add the location of these scripts at the 
start of PATH, thus overiding the compiler. Now all configures will 
default to 64 bit.

Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: threading support in python

[Daniel Dittmar]

Rob Williscroft wrote:
> Daniel Dittmar wrote in news:[EMAIL PROTECTED] in 
> comp.lang.python:
> 
> 
>>- removing reference counting and relying on garbage collection alone 
>>will break many Python applications (because they rely on files being 
>>closed at end of scope etc.)
>>
> 
> 
> They are already broken on at least 2 python implementations, so
> why worry about another one.

I guess few applications or libraries are being ported from CPython to 
Jython or IronPython as each is targeting a different standard library, 
so this isn't that much of a problem yet.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: threading support in python

[Daniel Dittmar]

km wrote:
> Is there any PEP to introduce true threading features into python's
> next version as in java? i mean without having GIL.
> when compared to other languages, python is fun to code but i feel its
> is lacking behind in threading

Some of the technical problems:

- probably breaks compatibility of extensions at the source level in a 
big way, although this might be handled by SWIG, boost and other code 
generators
- reference counting will have to be synchronized, which means that 
Python will become slower
- removing reference counting and relying on garbage collection alone 
will break many Python applications (because they rely on files being 
closed at end of scope etc.)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginner Textbook

[Daniel Dittmar]

M_M wrote:
> I am looking for a simple text book to introduce 13 to 18 year olds to 
> python programming. Suggestion?

If they're Germans: Python für Kids
http://www.amazon.de/gp/product/3826609514/028-9407382-2771748?v=glance&n=299956

or Python Programming for the Absolute Beginner
http://www.amazon.com/gp/product/1592000738/104-2712970-0839936?v=glance&n=283155
which seems to use a game as demo project.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CSV with comments

[Daniel Dittmar]

Sion Arrowsmith wrote:
> Daniel Dittmar  <[EMAIL PROTECTED]> wrote:
>> if line [:1] == '#':
> 
> What's wrong with line[0] == '#' ? (For one thing, it's fractionally
> faster than [:1].)
> 

Matter of taste. Occasionally, I use line iterators that strip the '\n' 
from the end of each line, so empty lines have to be handled. Of course, 
in my example code, one could have moved the check for the empty line 
before the check for the comment.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Partial classes

[Daniel Dittmar]

Sanjay wrote:
> Hi All,
> 
> Not being able to figure out how are partial classes coded in Python.
> 
> Example: Suppose I have a code generator which generates part of a
> business class, where as the custome part is to be written by me. In
> ruby (or C#), I divide the code into two source files. Like this:
> 
> GeneratedPerson.rb
> Class Person
>   .
>   .
>   .
> 
> End Class
> 
> HandcraftedPerson.rb
> Class Person
>  .
>  .
>  .
> End Class
> 
> The intrepretor adds the code in both to compose the class Person.
> 
> What is the equivalent in Python? Inheriting is a way, but is not
> working in all scenerios.

# HandcraftedPerson.py
import GeneratedPerson

class Person:
 def somemethod (self):
 pass


GeneratedPerson.Person.somemethod = Person.somemethod

Using reflection to merge all methods of HandcraftedPerson.Person into 
GeneratedPerson.Person is left as an exercise.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Coding style

[Daniel Dittmar]

Patrick Maupin wrote:
> The argument that one should always use len() to test whether sequences
> are empty or not simply because the use of len() gives a free "is it a
> sequence?" type-check is not apt to be well received by an audience
> which rejects explicit type-checking in general.

No explicit type check. More of an implicit behaviour check. OK,
wise guys are not apt to be well received by  ... well, any audience.

> The perverse wish, expressed in the specific example, that SOME piece
> of code SOMEWHERE should PLEASE throw an exception because some idiot
> passed a generator expression rather than a list into a function, is
> not apt to be well received by an audience which strives for generality
> when it makes sense; and I daresay most members of that audience, if
> confronted with the wished-for exception, would fix the function so
> that it quite happily accepted generator expressions, rather than
> changing a conditional to use len() just so that an equivalent
> exception could happen a bit earlier.

Premature generalization: the new 'premature optimization'.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CSV with comments

[Daniel Dittmar]

GinTon wrote:
> GinTon wrote:
>> In csv.reader, is there any way of skip lines that start whith '#' or
>> empty lines
>> I would add comments at my CSV file
> 
> For skip comment I get a dirty trick:
> 
> reader = csv.reader(open(csv_file))
> for csv_line in reader:
> if csv_line[0].startswith('#'):
> continue
> 
> But not possible let blank lines.
> 
> I think that CSV class should to let skip comments and new lines of
> auto. way.
> 

write an iterator that filters line to your liking and use it as input 
to cvs.reader:

def CommentStripper (iterator):
 for line in iterator:
 if line [:1] == '#':
 continue
 if not line.strip ():
 continue
 yield line

reader = csv.reader (CommentStripper (open (csv_file)))

CommentStripper is actually quite useful for other files. Of course 
there might be differences if a comment starts
- on the first character
- on the first non-blank character
- anywhere in the line

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using Visual Slick Edit for Python

[Daniel Dittmar]

Brian Buderman wrote:
> Anyone know if there is a way to make slickedit tag the built in modules 
> such as os and sys so that code completion and the like work?

I don't think that this is possible as the docs are in the python 
executable. But it should be easy to generate a dummy module:

import sys
for name, entry in sys.__dict__.iteritems ():
 if callable (entry):
 print '''def %s ():\n\t%s\n\tpass\n\n''' \
 % (name, entry.__doc__)

Put the generated modules in a directory outside of the PYTHONPATH, but 
add it to the Python tagging of SlickEdit.

Daniel

P.S. Have you already upgraded to v11 and is there anything worthwhile 
in it (Python or otherwise)?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to use ParseTuple with unknown number of args?

[Daniel Dittmar]

ondekoza wrote:
> The earlier posts generally highlight some specific feature of calling
> with arrays. Can someone point me to a piece of software, where sth.
> like this is actually used?

Python source/Objects/stringobject.c string_join would be an example if 
you only allow a list or tuple.

Python/bltinmodule.c min_max demonstrates how to allow either a sequence 
or a variable number of arguments.

Modules/itertools.c has probably lots of examples on how to work with 
iterators at the C API level.

Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: running python from a memory stick?

[Daniel Dittmar]

John Salerno wrote:
> Is there a way to 'install' and use Python on a memory stick, just as 
> you would on any computer? I use Windows, and I know the installation 
> does things with the registry, so probably I couldn't use the executable 
> file to install it. But is it possible to do it some other way, such as 
> how you might build it yourself on Linux (although I don't know how to 
> do that yet) and then just write and run scripts normally straight from 
> your memory stick?

Python looks for the libraries relative to python.exe. So you can copy 
the Python installation directory to your memory stick (make sure that 
python24.dll is included, this is copied to %windir%\system32 if you 
have done  the admin installation) and it should just work. You can test 
that by renaming in your registry 
HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.4 (or whatever version 
you're using)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Relying on the behaviour of empty container in conditional statements

[Daniel Dittmar]

horizon5 wrote:
> Hi,
> 
> my collegues and I recently held a coding style review.
> All of the code we produced is used in house on a commerical project.
> One of the minor issues I raised was the common idiom of specifing:
> 
> 
> if len(x) > 0:
> do_something()
> 
> Instead of using the language-defined bahviour, as stated by PEP8:
[...]
> Without wishing to start a flame war, what are other's opinions on
> this, is using "len" safer? If so why?

All objects evaluate to a boolean, but not all objects support len.
So if someone passes x = iter ([]) to your method, then len (x) will 
result in an exception, whereas 'if x:' will happily (and wrongly) 
answer true.

On the other hand, maybe you should be able to accept iterators, and 
then the test would look different anyway.

> 
> My own feeling is that I am willing to work with the behaviours defined
> by Python, and treat the use of len in these cases as excessive
> duplication (this is however, quite a minor point i agree).

I find

if parameters:

perfectly acceptable, as it allows parameters to be None as well.
On the other hand, I can't stand interpreting integers as booleans, so I 
wouldn't write

if count:

Everyone has their personal quirks.

> 
> Note that I have much more experience with the language (6-7 years),
> whilst the majority of my collegues have about 1-2 years experience.
> 

I guess they're coming from Java and not from Perl :-)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Questions about OSS projects.

[Daniel Dittmar]

bio_enthusiast wrote:
> I was wondering how to go about starting an open source project for
> doing routine biological problems? There is a plethora of scripts and
> a fairly large biopython project to back up anyone who tried, these
> however cater to the bioinformatics community and it loses the vast
> majority of the wet-lab scientists. How can someone who is used to
> writing small scripts and doing wet-lab work contribute to the open
> source community? Starting software projects seems to be the domain of
> people with much more experience and skill but there are some serious
> needs by people who do not have the skills to upkeep any software
> based project.
> 

If you've written a few small scripts that might be of use to others and 
that you assume that there are others who do the same, you might start 
with a wiki or something like the Python Cookbook 
(http://aspn.activestate.com/ASPN/Python/Cookbook/), but geared toward 
labs and biology.

If this gains any traction (that is if you get additional code snippets, 
people are commenting etc.), after a while, it might be useful to look 
at the material and see if there is enough code that warrants a library. 
This does not mean to simply package all the scripts into one package, 
but to see if there are any common tasks among the scripts to 'refactor' 
them into a library.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 'module' object has no attribute 'ssl'

[Daniel Dittmar]

Niurka Perez wrote:
> ssl = socket.ssl(sock, self.key_file,
> self.cert_file)
> AttributeError: 'module' object has no attribute 'ssl'

The socket module failed to import the _ssl module. And the ssl function 
gets only defined if _ssl could be imported.

You probably haven't installed the OpenSSL-dev rpm. When the Python 
configure couldn't find the SSL header files, it struck _ssl from the 
list of compilable modules.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Duplex communication with pipes - is possible ?

[Daniel Dittmar]

Dara Durum wrote:
> Now I trying with packet size decreasing. Are PIPE-s can handle the
> binary data packets, or I need to convert them with base64 ?

In the client, you need to set the mode of sys.stdin to binary, 
otherwise, you get the DOS translation of linefeeds. See
http://mail.python.org/pipermail/python-list/2000-January/020463.html
for some hints.

I assume that the stream returned by subprocess.popen are also opened in 
text mode and you have to change them tobinary, see the second hint in 
the link mentioned above.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XML, JSON, or what?

[Daniel Dittmar]

> My client-server is Python-to-Python. At present, I am using cPickle to
> transfer objects between the two. Among other things, I sometimes
> transfer a tuple. Using JSON it appears on the other side as a list. As
> I sometimes use the tuple as a dictionary key, this fails, as you
> obviously cannot use a list as a key.
[...]
> Can someone confirm this, or is there an easy workaround?

You can always convert a list to a tuple using the tuple () builtin 
right before  you use it as a key. But you have to be sure that it is a 
list. tuple ("abc") => ('a', 'b', 'c') is probably not what you intend 
to do.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Duplex communication with pipes - is possible ?

[Daniel Dittmar]

Dara Durum wrote:
> Hi !
> 
> I want to create a Process Pool Object.
> I can hold started processes, and can communicate with them.
> 
> I tryed with many ipc methods, but every of them have bug or other problem.
> Sockets are unavailabe (because Windows Firewall hold them).
> 
> I think I will use pipe.
> 
> The object's pseudocode:
> while not Quit:
> CheckProcessOutputs;
> ProcessReceivedData;
> SendDataToSubProcesses;
> if NoMoreData: Quit=1
> 
> If I used pipes and subprocess module in Windows, I got big freezes,
> and deadlocks.
> 
> Main proc:
> subprocpipe.write('\n')
> subprocpipe.readlines()
> 
> Sub proc:
> input=sys.stdin.readlines().strip()
> print input+'!!!'
> 
> It is working. But when I move this client code to cycle, I got deadlock.
> while not Quit:
> input=sys.stdin.readlines().strip()
> print input+'!!!'
> Quit=input=='q'
> 
> Why ? Why I cannot create cyclic communication with client ?
> Subprocess must "staying alive" (don't die), and need to stay in
> reuseable state ?
> 
> Simply: subprocess pool needed !!!
> 
> Thanks for help:
> dd

readlines () will try to read until the stream/socket is closed. Try to 
read only one line. This of course means that you cannot sent \n as part 
of the data, you have to escape them somehow.

I'm not sure how the pipe code is searching for the \n. Trying to read 
too much could lead to deadlocks as well. (Although I'm sure that the 
code is written to return fewerbytes than requested if there isn't 
enough data pending in the pipe). A safer variant might be to transfer a 
fixed number of bytes containing the length n of the following data, and 
then n bytes containing the actual data.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DB-API: how can I find the column names in a cursor?

[Daniel Dittmar]

A.M wrote:
> for row in out_cur:
> 
>  print row
> 
[...]

> 
> The other problem is accessing data in each row by column name. 

One useful technique is
for col1, col2, col3 in out_cur:
 sum = sum + col3

Access is still by index, but your code uses ordinary Python variables.
It works better with SELECTs as you can choose the output columns in the 
SQL. With stored procedures, there's always the possibility of someone 
changing the structure of the result set.

There exists a general library for your solution: 
http://opensource.theopalgroup.com/

Daniel


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is anybody knows about a linkable, quick MD5/SHA1 calculator library ?

[Daniel Dittmar]

DurumDara wrote:
> Hi !
> 
> I need to speedup my MD5/SHA1 calculator app that working on 
> filesystem's files.

You could try using threads. This would allow the CPU and the disk to 
work in parallel.

The sha/md5 modules don't seem to release the global interpreter lock, 
so you won't be able to use multiple CPUs/cores yet.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GUI issues in Python

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> Thanks Daniel, I really think that this should be the solution to my
> problem.
> 
> A quick Question...is wxPython Operating System dependent or it can be
> used with anu OS like Linux, Windows and Mac ?
> 

see http://www.wxpython.org/download.php#binaries

Linux and Win32 have been supported for a while. And the screenshot in 
my first reply was from Mac OS X, so OGL seems to be supported there too.

The wxPython demo application has an OGL example, so you can check that 
pretty quickly.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GUI issues in Python

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> I want to create a GUI where a user can select drag and drop kind of
> boxes, circles and make connections between them.
> 
> This is basically for depicting states and dependencies. I am writing a
> program where I let the user input states and dependencies in a certain
> domain. Based on input given by user in the GUI, I grab it and run my
> algorithm to generate the output.
> 
> I am wondering if there is any library like that in Python, or if
> anybody of you has done something similar in past, could you post some
> pointers.

wxPython has an interface to the OGL library (screenshot: 
http://www.wxpython.org/ss/wxPythonOSX.png, features: 
http://www.wxwindows.org/contrib/ogl/ogl.htm)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: HTMLParser fragility

[Daniel Dittmar]

Lawrence D'Oliveiro wrote:
> I've been using HTMLParser to scrape Web sites. The trouble with this 
> is, there's a lot of malformed HTML out there. Real browsers have to be 
> written to cope gracefully with this, but HTMLParser does not. Not only 
> does it raise an exception, but the parser object then gets into a 
> confused state after that so you cannot continue using it.
> 
> The way I'm currently working around this is to do a dummy pre-parsing 
> run with a dummy (non-subclassed) HTMLParser object. Every time I hit 
> HTMLParseError, I note the line number in a set of lines to skip, then 
> create a new HTMLParser object and restart the scan from the beginning, 
> skipping all the lines I've noted so far. Only when I get to the end 
> without further errors do I do the proper parse with all my appropriate 
> actions.

You could try HTMLTidy (http://www.egenix.com/files/python/mxTidy.html) 
as a first step to get well formed HTML.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and microsoft outlook-using com, can I interact with msoutlook?

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> Hi All,
> I know that Microsoft Exchange has a com interface, CDO, but I can't seem to
> find one for Microsoft outlook.
> does anyone have code snippets for using msoutlook and python, or
> suggestions?

You can use CDO to manage your Inbox, send mail etc.

The following functions navigate to a certain folder and parse every 
message text for a certain regular expression:

def spamstat ():
 s = Dispatch ("Mapi.Session")
 s.Logon ("Default Outlook Profile")
 junk = findJunk (s)
 rex = re.compile ('The ([0-9]+) emails listed below')
 for msg in junk.Messages:
 text = msg.Text
 match = rex.search (text)
 if match:
 date = parseDate (msg.TimeSent)
 print (date, int (match.group (1)))

def findJunk (s):
 inbox = s.Inbox
 for folder in s.Inbox.Folders:
 if folder.Name == 'Junk':
 return folder


See the Spambayes Outlook plugin (http://spambayes.sourceforge.net/) for 
a complex example.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: import random module

[Daniel Dittmar]

DataSmash wrote:
> Hi,
> When I import the random module at the python interpreter, it works
> fine:
 import random
 x = random.randint(1,55)
 print x
> 14
> 
> BUT, when I put the same code in a python script:
> * random.py:
> 
> import random
> 
> x = random.randint(1,55)
> print x
> 
> and run it at the command line, I get:
> Traceback (most recent call last):
>File p:\temp\random.py, line 7, in ?
>   import random
>File p:\temp\random.py, line 8, in ?
>   x = random.randint(1,55)
> AttributeError: 'module" object has no attribut 'randint'
> 
> I run scripts at the command line everyday so there must be something
> specifically
> wrong with the "random" module, unless I'm totally missing something
> here.

Just put a
print random
after the
import random.py

You'll probably see that Python is importing your main module instead of 
the one from the lib.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: global, globals(), _global ?

[Daniel Dittmar]

Roy Smith wrote:
> In article <[EMAIL PROTECTED]>,
>  robert <[EMAIL PROTECTED]> wrote:
> 
>> Using global variables in Python often raises chaos. Other languages use 
>> a clear prefix for globals.
> 
> Unsing globals raises chaos in any language.  They should be shunned and 
> avoided.

Solution: replace globals with Singletons ;-)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Module written in C does not repond to Ctrl-C interruption.

[Daniel Dittmar]

Bo Peng wrote:
> Daniel Dittmar wrote:
> 
>>
>> You could set up your own signal handler when entering the C 
>> extension. This should abort the extension (tricky) and call the 
>> Python signal handler.
> 
> 
> This can be done under linux using things in signal.h but I am not
> sure whether or not there is a portable way to do it (max, win32).
> Does anybody know a quick way under windows?


SetConsoleCtrlHandler:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/setconsolectrlhandler.asp

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Module written in C does not repond to Ctrl-C interruption.

[Daniel Dittmar]

Diez B. Roggisch wrote:
> """
> Although Python signal handlers are called asynchronously as far as the
> Python user is concerned, they can only occur between the ``atomic''
> instructions of the Python interpreter. This means that signals arriving
> during long calculations implemented purely in C (such as regular
> expression matches on large bodies of text) may be delayed for an arbitrary
> amount of time. 
> """
> 
> So - no workaround here, unless you patch python.

You could set up your own signal handler when entering the C extension. 
This should abort the extension (tricky) and call the Python signal handler.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using repr() with escape sequences

[Daniel Dittmar]

nummertolv wrote:
> - Consider a string variable containing backslashes.
> - One or more of the backslashes are followed by one of the letters
> a,b,f,v or a number.
> 
> myString = "bar\foo\12foobar"
> 
> How do I print this string so that the output is as below?
> 
> bar\foo\12foobar
> 
> typing 'print myString' prints the following:
> 
> baroo
> foobar
> 
> and typing print repr(myString) prints this:
> 
> 'bar\x0coo\nfoobar'
> 

The interpretation of escape sequences happens when the Python compiler 
reads the string "bar\foo\12foobar". You'll see that when you do 
something like
 >>> map (ord, "bar\foo\12foobar")
[98, 97, 114, 12, 111, 111, 10, 102, 111, 111, 98, 97, 114]
This displays the ASCII values of all the characters.

If you want to use a string literal containing backslashes, use r'' strings:
 >>> myString = r'bar\foo\12foobar'
 >>> map (ord, myString)
[98, 97, 114, 92, 102, 111, 111, 92, 49, 50, 102, 111, 111, 98, 97, 114]
 >>> print myString
bar\foo\12foobar
 >>> print repr (myString)
'bar\\foo\\12foobar'

If you get the strings from an external source as suggested by your 
original post, then you really have no problem at all. No interpretation 
of escape sequences takes place when you read a string from a file.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Add a month

[Daniel Dittmar]

Fredrik Lundh wrote:
> [EMAIL PROTECTED] wrote:
> 
>> Hi, this is probably a really simple question but...
>> How do you add a month to a datetime date in python? It would be nice
>> if  you could do something like:
>>
>> d = datetime.date(2006,2,17)
>> dm = datetime.timedelta(months=1)
>> d_new = d + dm
>>
>> but timedelta doesn't have a 'months' setting. Am I missing some easy
>> method or do I have to code a work around myself??
> 
> what do you expect d_new to be after the operation ?  if the answer
> is date(2006,3,17), what's date(2006,1,31) plus one month?

Traceback (most recent call last):
   File "", line 1, in ?
ValueError: date domain error

in analogy to math.sqrt (-2)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: low level data types

[Daniel Dittmar]

dementrio wrote:
> Thanks for the hint!
> 
> However now I have another problem - endianness (the client runs on
> powerpc, the server on x86). I found that simply reversing the stuff I
> recv() works, but is there any cleaner way for taking care of this?
> 

http://docs.python.org/lib/module-struct.html
and search for "Alternatively, the first character of the format string 
can be used to indicate the byte order"

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sharing database connection from C to Python

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> I'm developing an application using the C language and Python for it's
> plugins. The C program connects to a MySQL database and keeps that
> connection active. Is it possible to 'share' this connection with the
> Python plugins? If so, is there a "standard" way to do that?

There is no simple way as MySQL connections in Python are Python 
extension objects. You'll probably have to use a custom MySQL driver.

Either create the session in C and add a constructor to the python to 
create a MySQL connection object from that handle or pointer

Or add a method to the Python MySQL object to get the handle and 
pointer. Create all database connections in Python (precise: using the 
Python API from C), get the handle and pointer to use it from C, pass 
the original session to plugins.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DB API and thread safety

[Daniel Dittmar]

Robin Haswell wrote:
> Hey guys
> 
> I've been reading http://www.python.org/peps/pep-0249.html and I don't
> quite get what level of thread safety I need for my DB connections.
> 
> If I call db = FOOdb::connect() at the start of my app, and then every
> thread does it's own c = db.cursor() at the top, what level of thread
> safety do I need to avoid threads stepping on each other? Hopefully the
> answer to this question will get me oriented enough to understand the
> other options :-)

Even if you can share the connection between threads, it will be the 
rare database which can actually work concurrently on one connection. 
Most drivers promising "threads may share the module and connections" 
will use a lock so that only one operation is active at a time.

For maximum throughput (at the cost of using more ressources on the 
database), you'd use one connection per thread. If you want to use 
transactions, then it is the only way.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple modules with database access + general app design?

[Daniel Dittmar]

Frank Millman wrote:
> I have subclassed threading.Thread, and I store a number of attributes
> within the subclass that are local to the thread. It seems to work
> fine, but according to what you say (and according to the Python docs,
> otherwise why would there be a 'Local' class) there must be some reason
> why it is not a good idea. Please can you explain the problem with this
> approach.

Your design is just fine. If you follow the thread upwards, you'll 
notice that I encouraged the OP to pass everything by parameter.

Using thread local storage in this case was meant to be a kludge so that 
not every def and every call has to be changed. There are other cases 
when you don't control how threads are created (say, a plugin for web 
framework) where thread local storage is useful.

threading.local is new in Python 2.4, so it doesn't seem to be that 
essential to Python thread programming.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple modules with database access + general app design?

[Daniel Dittmar]

Robin Haswell wrote:
> Ah I see.. sounds interesting. Is it possible to make any module variable
> local to a thread, if set within the current thread? 

Not directly. The following class tries to simulate it (only in Python 2.4):

import threading

class ThreadLocalObject (threading.local):
 def setObject (self, object):
 setattr (self, 'object', object)

 def clearObject (self):
 setattr (self, 'object', None)

 def __getattr__ (self, name):
 object = threading.local.__getattribute__ (self, 'object')
 return getattr (object, name)

You use it as:

in some module x:

db = ThreadLocalObject ()

in some module that create the database connection:

import x

def createConnection ()
 localdb = ...connect (...)
 x.db.setObject (localdb)

in some module that uses the databasse connection:

import x

def bar ():
 cursor = x.db.cursor ()

The trick is:
- every attribute of a threading.local is thread local (see doc of 
module threading)
- when accessing an attribute of object x.db, the method __getattr__ 
will first retrieve the thread local database connection and then access 
the specific attribute of the database connection. Thus it looks as if 
x.db is itself a database connection object.

That way, only the setting of the db variable would have to be changed.

I'm not exactly recommneding this, as it seems very error prone to me. 
It's easy to overwrite the variable holding the cursors with an actual 
cursor object.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple modules with database access + general app design?

[Daniel Dittmar]

Robin Haswell wrote:
> On Thu, 19 Jan 2006 14:37:34 +0100, Daniel Dittmar wrote:
>>If you use a threading server, you can't put the connection object into 
>>the module. Modules and hence module variables are shared across 
>>threads. You could use thread local storage, but I think it's better to 
>>pass the connection explicitely as a parameter.
> 
> 
> Would you say it would be better if in every thread I did:
> 
>   m = getattr(modules, module)
>   b.db = db
> 
>   ...
> 
>   def Foo():
>   c = db.cursor()
> 

I was thinking (example from original post):

import modules
modules.foo.Bar(db.cursor ())

 # file modules.foo
 def Bar (cursor):
 cursor.execute (...)

The same is true for other objects like the HTTP request: always pass 
them as parameters because module variables are shared between threads.

If you have an HTTP request object, then you could attach the database 
connection to that object, that way you have to pass only one object.

Or you create a new class that encompasses everything useful for this 
request: the HTTP request, the database connection, possibly an object 
containing authorization infos etc.

I assume that in PHP, global still means 'local to this request', as PHP 
probably runs in threads under Windows IIS (and Apache 2.0?). In Python, 
you have to be more explicit about the scope.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple modules with database access + general app design?

[Daniel Dittmar]

Robin Haswell wrote:
> cursor for every class instance. This application runs in a very simple
> threaded socket server - every time a new thread is created, we create a
> new db.cursor (m = getattr(modules, module)\n m.c = db.cursor() is the
> first part of the thread), and when the thread finishes all its actions
> (of which there are many, but all sequential), the thread exits. I don't

If you use a threading server, you can't put the connection object into 
the module. Modules and hence module variables are shared across 
threads. You could use thread local storage, but I think it's better to 
pass the connection explicitely as a parameter.

> separate connection, but I get the feeling that a lot of cursors = a lot
> of connections. I'd much prefer each method call with a thread to reuse
> that thread's connection, as creating a connection incurs significant
> overhead on the MySQL server and DNS server.

You can create several cursor objects from one connection. There should 
be no problems if you finish processing of one cursor before you open 
the next one. In earlier (current?) versions of MySQL, only one result 
set could be opened at a time, so using cursors in parallel present some 
problems to the driver implementor.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Windows and python execution

[Daniel Dittmar]

Tim Roberts wrote:
> Mark Carter <[EMAIL PROTECTED]> wrote:
>>What you need to do is include the following line in autoexec.bat:
>>set .py=c:\python24\python.exe
>>
>>This will achieve the desired result. I'm suprised more people don't use it.
> 
> 
> They don't use it, because it doesn't do anything.  I'd be interested to
> know where you read that.

This is a feature of 4NT, a commercial replacement for cmd.exe. And 
unlike cmd.exe, it even allows redirection of stdin/stdout when a script 
is executed through the extension (the OP's next post is probably going 
to be 'why doesn't redirection work?').

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Aproximative string matching

[Daniel Dittmar]

javuchi wrote:
> I'm searching for a library which makes aproximative string matching,
> for example, searching in a dictionary the word "motorcycle", but
> returns similar strings like "motorcicle".
> 
> Is there such a library?
> 

agrep (aproximate grep) allows for a certain amount of errors and there 
exist Python bindings (http://www.bio.cam.ac.uk/~mw263/pyagrep.html)

Or google for "agrep python".

Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: initialising a list of lists

[Daniel Dittmar]

Peter Kleiweg wrote:
> This does not what I want it to do:
> 
> >>> a = [[]] * 6
> >>> a[3].append('X')
> >>> a
> [['X'], ['X'], ['X'], ['X'], ['X'], ['X']]
> 
> This does what I want:
> 
> >>> b = [[] for _ in range(6)]
> >>> b[3].append('X')
> >>> b
> [[], [], [], ['X'], [], []]
> 
> The first is clear and wrong. The second is hairy and right.

> Is there a way to do it clear 

Define a function:

import copy

def init_list (count, element):
 return [copy.copy (element) for i in xrange (count)]

> and right?

Test it.

Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: D foreach

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> The Digital Mars D compiler is a kind of "improved c++", it contains a
> "foreach" statement:
> http://www.digitalmars.com/d/statement.html#foreach
> 
> Usage example:
> foreach(int i, inout int p; v1) p = i;
> 
> Is equal to Python:
> for i in xrange(len(v)): v[i] = i
[...]
> So the variable p contains (scans) the elements of the given iterable
> object, but if you assign p with a value, that value becomes copied
> inside the mutable iterable too. Those are little examples, but I think
> it can be quite useful in more complex code.

1. It would be difficult to implement. Python would require the concept 
of 'reference to variable', which has lots of repercussions for 
reference counting, garbage collection etc. Requiring iterators to 
return references would also break all existing iterators. It would also 
be required that the assignment operator is overloadable and this is 
another box of Pandora no one likes to open (well, no one except C++ 
programmemrs).

Or the compiler would have to detect 'assignment to iterator variable' 
and issue an 'update_current' to the iterator.

2. It violates the  Zen of Python 'Explicit is better than implicit' 
(although the definition of 'explict' varies wildly in the Python community)

3. For specific collections like lists and dictionaries, you could write 
a wrapper so that it is possible to write

for ref in wrapper (mycollection):
 print ref.value
 ref.value = newvalue

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why the nonsense number appears?

[Daniel Dittmar]

Johnny Lee wrote:
print time1, time2
> 
> 1130748744.461 1130748744.500
> 
float(time2) - float(time1)
> 
> 0.03934332275391
> 
> 
>Why are there so many nonsense tails? thanks for your help.

http://en.wikipedia.org/wiki/Floating_point#Problems_with_floating-point, 
especially 'Rounding'. Or google for "gloating point precision" if you 
need more details.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Absolultely confused...

[Daniel Dittmar]

Jeremy Moles wrote:
> So, here is my relevant code:
> 
>   PyArg_ParseTuple(args, "O!", &PyType_vector3d, &arg1)
> 
> And here ismy error message:
> 
>   argument 1 must be pylf.core.vector3d, not pylf.core.vector3d
> 

It looks as if two PyType_vector3d exist in your system
- the one that created the object passed to your routine
- the one in your extension code

As PyType_vector3d probably comes from a shared object/DLL
- does your code accesses really the same shared object that is also 
loaded by the Python interpreter? It could be that you linked with a 
specific file, but Python loads something different from $PYTHONPATH
- on Windows, you couldn't simply import a variable from a DLL, you had 
to call a special routine to get the pointer

One possible portable solution: in your module initialization
- import pylf.core
- create an object of type vector3d
- use your knowledge about the inner structure of Python objects and get 
the pointer to the PyType from the object
- store it in a module static variable TypeVector3D
- pass that variable to PyArg_ParseTuple

Browse the Python Extension API, maybe partts or all of this are already 
available.

There's still a problem left when pylf.core gets reloaded (rare, but 
possible). I assume the shared object also gets reloaded, which means 
that the type objects gets loaded to a new address and PyArg_ParseTuple 
will complain again. I'm not sure if there is a solution to this, 
because there still could be objects create from the old module.

Maybe you should just check the type yourself by comparing the class 
names buried in the PyType. You could cache one or two type pointers to 
speed this up.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about smtplib, and mail servers in general.

[Daniel Dittmar]

Chris Dewin wrote:
> Hi. I've been thinking about using smtplib to run a mailing list from my 
> website.
> 
> s = smtplib.SMTP("server")
> s.sendmail(fromaddress, toaddresess, msg)
> 
> I know that in this instance, the toaddresses variable can be a variable
> of type list.
> 
> Suppose the list contains well over 100 emails. Would that create some
> sort of drain on the mail server? Would I be better off doing it in some
> other way? 
> 

Not really an answer to your question, but it's probably considered bad 
style to publish the email addresses of the recipients via the address 
list. Use a neutral To-address (best: the mailing list address) and add 
the recipients via bcc: headers.

You might also want to look at mailman 
http://www.gnu.org/software/mailman/, which is a complete mailing list 
solution written in Python.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Windows paths, Java, and command-line arguments, oh my!

[Daniel Dittmar]

Steve M wrote:

About your main problem: I'm still convinced that it's the order of -jar 
and -D that is important, see my other post.

> I have tried (not entirely systematically but pretty exhaustively)
> every combination of backslashes in the cmd string, e.g.:
>  -Dsalesforce.config.dir=c\:\\config
>  -Dsalesforce.config.dir=c:\\config
>  -Dsalesforce.config.dir=c\\:\config
>  -Dsalesforce.config.dir=c\\:\\config
>  etc.

A hint:
- if you're unsure how something must be entered as a literal, test it 
in the interactive interpreter:

 >>> raw_input ('enter a path: ')
enter a path: c:\config
'c:\\config'

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Windows paths, Java, and command-line arguments, oh my!

[Daniel Dittmar]

Steve M wrote:
> I'm trying to invoke a Java command-line program from my Python program
> on Windows XP. I cannot get the paths in one of the arguments to work
> right.
> 
> The instructions for the program describe the following for the
> command-line arguments:
> 
> java -jar sforcedataloader.jar -Dsalesforce.config.dir=CONFIG_DIRECTORY
> 
> They also give an example:
> 
> java -Dsalesforce.config.dir=c:\config -jar sforcedataloader.jar
> 
> If I type the example above at the cmd.exe command line the thing works
> (assuming I have the config file in c:\config). What doesn't work is
> these two lines:
> 
> cmd = r'java -jar sforcedataloader.jar -Dc:\config'
> os.system(cmd)

If you write
java -jar x.jar -Dwhatever=x
then -Dwhatever=x is passed as an argument to the main method of the 
main class in x.jar.

If you write
java -Dwhatever=x -jar x.jar
then -Dwhatever=x is interpreted by java and put into the system properties.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to write this iterator?

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> Given a list of iterators, I'd like to have a new one that would 
> cyclically walk over the list calling the next() method of the iterators 
> (removing any iterator which is exhausted). It should also support adding 
> a new iterator to the list; it should be added in front of the current 
> position (so that it's called only after all the others). This is what I 
> was able to write with my zero python skills (self.iters is the list of 
> iterators, self.i is the index of the iterator that should be used next). 
> Is there a nicer/more pythonic solution, maybe using generators?

A generator version:

def iterconcat (collectionlist):
 for collection in collectionlist:
 for element in collection:
 yield element

Extending collectionlist at runtime will work only with lists and 
similar collections. And you'll need to keep a reference to it as well. 
And it would be called after all the others, which matches your 
description, but not your code.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: packaging python for install.

[Daniel Dittmar]

J wrote:
> I have created an App that embedds the python interpreter and I am
> now in the process of creating an installer. I am currently linking
> python24.lib, but it is only 184k and I suspect that it imports other
> dlls... I am also using numarray. Does anyone have any experiences in
> packaging python with an application in a single installer ? Has that
> been done before ?

- you need python24.dll (probably in %windir%\system32)
- you need some of the *.pyd found in you Pythondir\DLLs. You can use 
Proces Explorer from www.sysinternals.com to see all the loaded DLLs and 
.pyd that your running program accesses
- you need a large part of the Python library. Best: compile them to 
.pyc files using Pythondir\Lib\compileall.py, pack them into one .zip 
file and then add that .zip file to %PYTHONPATH%

Hint: Python will search for the Library files through the registry key 
HKEY_LOCAL_MACHINE\SOFTWARE\PythonCore\. You don't have to set 
this entry with your installer (and you shouldn't, as it would break an 
already existing Python installation). But you could rename it on your 
machine as this allows you to test your installation without having to 
copy everything to a clean machine.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: re module help

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> if I start replacing regex by re I get stuck at replacement of
>regex.symcomp() and regex.pattern()

Groups identified by names are part of the standard regular expression 
syntax:
regex.symcomp ('\([a-z][a-z0-9]*\)')
becomes
re.compile ('(?Pid[a-z][a-z0-9]*)')

I get AttributeError for both regex.pattern and regex.compile 
('abc').pattern.

re compiled regular expression have a pattern attribute, this was named 
givenpat and realpat in the regex module.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: warning when doubly liked list is defined globally

[Daniel Dittmar]

chand wrote:
> Hi.,
> 
> In my api.py file 'g_opt_list' is defined globally
> g_opt_list =[[],[],[],[],[],[],[]]
> 
> when I run the py file, I am getting the Following Error
> 
> SyntaxWarning: name 'g_opt_list' is used prior to global declaration

g_del_opt_list =[[],[],[],[],[],[],[]]
#g_opt_list = []

It seems that your variable names got mixed up.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what's the difference between *.dll and *.pyd if both of them are extended python module?

[Daniel Dittmar]

wen wrote:
> 1. what's the difference between them? i saw the code is same as common c++
> extended python module, can i use microsoft visual C++ to create a dll
> project for compiling it as _cmd.dll?

.pyd is just a naming convention. It was probably introduced to prevent 
name clashes with unrelated DLLs.

> 2. i have never written a module with extension *.pyd, how to make a *.pyd?
> and, i doubt, how the author debug the _cmd.pyd, only by Makefile?

I think you can configure the output file name in the project properties.

> 3. i have a common problem about writting a extend module by c/c++, i often
> use msvc to write it, but i only can invoke release dll, not debug dll, even
> i use command like this: python_d test.py, in which case, test.py import
> test.dll, and invoke test.hello() api. why? anybody has nice way to debug
> it? thank you.

Assuming your module is called _cmd:
- add a function init_cmd_d to the extension module (init_cmd_d simply 
calls init_cmd)
- import cmd_d in your Python script

I must admit I haven't tried this, it is based on my understanding of 
how Python loads extension modules.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loop in python

[Daniel Dittmar]

km wrote:
> Why is it that the implementation of empty loop so slow in python when 
> compared to perl ? 
[...]
> Is python runtime slow at all aspects when compared to perl ? 

No

> I really wonder what makes python slower than perl ? 

It could be that the Perl compiler recognizes such a for loop and emits 
special code (e.g. an integer loop variable that gets incremented). 
Python creates an xrange object and then has the overhead of 1000 next() 
calls.

You should benchmark other features before jumping to conclusions: calls 
to functions, calls to methods, calls to inherited methods.

Others will probably chime in with arguments whether such micro 
benchmarks are a useful indication of the speed a complete program at all.

> Is there any proposal to make python faster in future versions ? 

Yes in the general case, probably no in this specific case.


Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len(sys.argv) in (3,4)

[Daniel Dittmar]

Daniel Schüle wrote:
> if __name__ == "__main__":
> if len(sys.argv) not in (3,4):
> print "usage: prog arg1 argv2 [-x]"
> # etc ...
> 
> while develeoping I had my interpeter running and reloaded module
> now since I am ready, I wanted to run it from cmd windows shell
> but it always prints "usage ..."
> I added print len(sys.arg) and it prints 1 though I am provinding more
> than 1 value

Add

import sys
print sys.argv

at the top of the script before any other import.
Maybe one of the modules you're importing messes with sys.argv

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to write thread-safe module ? and pytz

[Daniel Dittmar]

nicolas_riesch wrote:
> Does someone know if the module pytz
> (http://sourceforge.net/projects/pytz/) is thread-safe ?
> I have not seen it explicitely stated, and just wanted to be sure, as I
> want to use it.
> 
> That's because in the file pytz/tzinfo.py, I see global variables
> _timedelta_cache, _datetime_cache, _ttinfo_cache, which are
> dictionnaries and are used as cache.
> I always thought that you must protect shared data with locks when
> multithreading, but I don't see any lock anywhere in pytz.

Dictionaries (and probably most other Python types that are implemented 
in C) are inherently thread safe.

This applies only  to the individual methods of dictionaries. The 
following code would still require a lock:
if mydict.has_key (keyval):
 variable = mydict [keyval]
because a second thread could delete the entry between the calls to 
has_key and __getvalue__.

mydict [keyval] = mydict.get (keyval, 0) + 1
is also an candidate for problems.

> However, pytz seems to work well with multiple threads creating various
> timezone objects at the same time.

'Seems to work' is never a good argument with regard to threads. 
Especially if you're testing on a single CPU machine.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: API class creation

[Daniel Dittmar]

kman3048 wrote:
> Hello,
> 
> as a relative newcomer to Python API programming I've got a problem:
> 
> To extend Python:
> - there is an API C call to create a module
> - there is also a API C call to create a method
> - there is an API C call to create a Class instance
> 
> Now, I need to create a Class and fill it with Methods and Variables.
> There are means to create (and attache) methods and variables. 
> However, I have not found how to create a Class within a Module. Or do 
> I have to use a low level API function to allocate an Object from Heap?

static PyMethodDef moduleMethods [] = {
 ...
};

statichere PyTypeObject MyClassType = {
 PyObject_HEAD_INIT (NULL)
 ...
};

initmymodule ()
{
 PyObject* module;
 PyObject* dict;

 module = Py_InitModule4 ("mymodule", moduleMethods,
   "doc string", NULL, PYTHON_API_VERSION);
 if (module == NULL) {
 return;
 }
 dict = PyModule_GetDict (module);
 PyDict_SetItemString (dict, "MyClass"),
 (PyObject*) &MyClassType));
}

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dabo in 30 seconds?

[Daniel Dittmar]

Jorge Godoy wrote:
> I agree where you say that lack of information is a risk.  But I don't see
> how it -- lack of information -- wouldn't affect both scenarios.  At least,

Because you put different probabilities on different outcomes. One easy 
'risk markup' would be to assume that parts of the standard Python 
distribution like TkInter have a higher chance of working with the next 
release than external libraries like wxPython. Of course, there are lots 
of other possible criteria:
- which has been under more active development in the last releases
- which source code is easier to understand so that I don't have to rely 
on external help

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dabo in 30 seconds?

[Daniel Dittmar]

Jorge Godoy wrote:
> We can find several problems, almost all of them can be solved with the
> admin's creativity.  

You must distinguish between solving technical problems once a course 
has ben set and choosing such a course in the first place.

The latter has to deal also with the risks of the unknown. Of course 
what is unknown can be influenced somewhat by getting information.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dabo in 30 seconds?

[Daniel Dittmar]

Cliff Wells wrote:
> I can understand this, but from my experience, their concerns are badly
> misplaced:  I recently wrote a fairly sizable Python app (~8K LOC) that
> utilized several 3rd party python librarys: wxPython, Twisted,
> FeedParser, DateUtils and SQLite to name a few off the top of my head
> (plus I had to repackage libxml and libxslt on OS/X because 10.3 ships
> with broken versions :P). 
> 
> It ran on Windows and OS/X (and Linux, but that was never deployed as
> the customer wasn't interested).  This was for a *very* large customer
> and made it to nearly 10,000 desktops.  Not one complaint had to do with
> installation of 3rd party packages.  Why?  Because I *packaged* it for

Actual success stories like this one are often more convincing than 
general remarks about how any problem is solvable, given enough effort.

> In short, these people's complaints reveal only two things: 1) they are
> hopelessly pessimistic, whether out of pure laziness, lack of experience

It might be very specific experiences. Many people using third party 
controls with Visual Basic got burned.

> endeavor.  As I mentioned earlier, programming is half brains and half
> tenacity.  

I'd add a bit of gambling, because often, you don't have enough 
information. Telling what works from your experience adds useful 
information so that the brain can decide where to invest the tenacity.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dabo in 30 seconds?

[Daniel Dittmar]

Jorge Godoy wrote:
> Daniel Dittmar wrote:
> 
> 
>>To be fair to those slothes: some of them want to write software for a
>>commercial setting where they have to install it on other peoples
>>machines. So it isn't just getting it to work one one own's machine.
>>Using a specifc Python library with external dependencies means also
>>installing and *supporting* it on a possible large set of configurations.
> 
> 
> I see no problem with that.  Specially since there are lots of ways to share
> directories on a network installation.  You install it once and it's done.
> 

Some on Windows, some on one Linux, some on another Linux with a newer 
GTK, some want it on their laptops to work on the road ...

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dabo in 30 seconds?

[Daniel Dittmar]

Cliff Wells wrote:
>  But then I'm willing to
> actually work a little to get what I want.  For other it seems they
> won't be happy unless you drive to their house and install it for them

To be fair to those slothes: some of them want to write software for a 
commercial setting where they have to install it on other peoples 
machines. So it isn't just getting it to work one one own's machine. 
Using a specifc Python library with external dependencies means also 
installing and *supporting* it on a possible large set of configurations.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Ten Essential Development Practices

[Daniel Dittmar]

Dark Cowherd wrote:
> There should be one-- and preferably only one --obvious way to do it.
> 
> But this not true of Python. 
> GUI, Web development, Application Framework - it is shambles. It is so

That's because there is no *obvious* way to do these.

> -Quote - Phillip J. Eby from dirtsimple.org
> Python as a community is plagued by massive amounts of
> wheel-reinvention. The infamous web framework proliferation problem is
> just the most egregious example.

In stark contrast to Java, where everybody uses standard components like 
JSP, Struts, Tapestry, JSF, Spring, ...

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rich Graphics?

[Daniel Dittmar]

Chris Spencer wrote:
> I'm trying to write a Gui in Python for manipulating rich graphical 
> representations, similar to something like Inkscape. I've tried tkinter, 
> wxPython, pyGtk, and while they all do traditional widgets well enough, 
> none of them really handle anti-aliased, transparent, transformed shapes 
> typical of vector based displays. I've noticed tkZinc, which seems to 
> better handle vector graphics through the use of openGL, but it's 
> traditional widget set is still limited and based on tkinter. Ideally, 
> what I'm looking for is something like wxWidgets which can display SVG 
> along side standard widgets and allow there manipulation through Python. 
> I was thinking of a web-based route, by accessing the SVG capabilities 
> in Mozilla's Deer Park browser through Twisted+Livepage, but this has 
> been extremely complicated and limiting. Are there any other options I 
> haven't considered?
> 
> Sincerely,
> Chris

maybe PyGame: http://www.pygame.org/, although it might be lacking in 
the standard widget department.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Daniel Dittmar]

Terry Reedy wrote:
> for dir in pathobject:
>   if isdir(dir): cd(dir)
> 
> *is*, in essence, what the OS mainly does with paths (after splitting the 
> string representation into pieces).

That's why there is rarely a need to to it in Python code.

> Directory walks also work with paths as sequences (stacks, in particular).

I'd say it works with stacks of pathes, not with stacks of path elements.

I'm not saying that there isn't any use for having a list of path 
elements. But it isn't that common, so it should get an methodname to 
make it more explicit.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Daniel Dittmar]

John Roth wrote:
> However, a path as a sequence of characters has even less
> meaning - I can't think of a use, while I have an application

That's true. But the arguments for path objects as strings go more in 
the direction of using existing  functions that expect strings.

> where traversing a path as a sequence of path elements makes
> perfect sense: I need to descend the directory structure, directory
> by directory, looking for specific files and types.

But then your loop doesn't need the individual path elements, but rather 
  sub-path objects

for p in pathobj.stepdown ('/usr/local/bin'):
 if p.join (searchedFile):
 whatever

I'm not saying that there isn't any use for having a list of path 
elements. But it isn't that common, so it should get an methodname to 
make it more explicit.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Extending python - undefined symbol error on import

[Daniel Dittmar]

ch424 wrote:
> However, when I open up the python command line, and type "from gpib
> import *" or "import gpib" I get "ImportError: /usr/.../gpibmodule.so:
> undefined symbol: ibdev" -- but I know it's defined in the ni488.h
> file, especially as I can use this code from actual C programs without
> problems. The ni488.h file in in the right place to be used for C
> compilation.

I guess ibdev is *declared* in ni488.h and you'll have to add the lib 
where it is *defined*. You'll probably have to add a
libraries = ['gpip']
to setup.py.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Daniel Dittmar]

Duncan Booth wrote:
>  I would have expected a 
> path object to be a sequence of path elements rather than a sequence of 
> characters. 

Maybe it's nitpicking, but I don't think that a path object should be a 
'sequence of path elements' in an iterator context.

This means that

for element in pathobject:

has no intuitive meaning for me, so it shouldn't be allowed.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: is a file open ?

[Daniel Dittmar]

luis wrote:
> for root, dirs, files in os.walk(path):
>for file in files:
>   # ¿ is opened ?

On Linux and some other Unixes, you can probably read the /proc filesystem.

On Windows, you'll probably get the quickest result by running 
handle.exe (http://www.sysinternals.com/Utilities/Handle.html).

Either way, the information you'll get is restricted by your permissions.

Either information will get stale really fast, so it's not suitable if 
your task is something like 'can I backup this directory or is someone 
writing to a file?'

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: automatic form filling

[Daniel Dittmar]

> I would like to know how I could automatically fill a
> (search) form on a web page and download the resulting
> html page. 

http://wwwsearch.sourceforge.net/ClientForm/


Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyo contains absolute paths

[Daniel Dittmar]

David Siroky wrote:
> When I "compile" my python files with "python -OO " into pyo files
> then they still contain absolute paths of the source files which is
> undesirable for me. How can I deal with that?

Use the script compileall.py (in Lib) to compile all the files. This 
script has an option -d  that allows to set the directory name 
compiled into the .pyo files.

 From the help: "-d destdir: purported directory name for error messages"

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Use cases for del

[Daniel Dittmar]

Scott David Daniels wrote:
> Testing for None should be an is-test (not just for speed). In
> older pythons the == result wasn't necessarily same as is-test.
> This made sense to me after figuring out NULL in database stuff.

NULL in SQL databases has nothing to do with Python None. I'm quite sure 
that None == None always returned boolean true.

In a SQL database, NULL = NULL will always return NULL, which is prety 
much the same as FALSE. Except for NOT, AS NOT NULL is NULL.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python nested class

[Daniel Dittmar]

Vedanta Barooah wrote:
> in a python nested class is it possible to change the value of the
> parent class's variable without actually creating an instance of the
> parent class

Python nested classs are like *static* Java nested classes. Non-static 
Java classes are very different in that they have an implicit reference 
to an instance of the enclosing class. This is why you can instantiate 
non-static inner classes only in the context (= non-static method) of a 
specific object. There is no direct equivalent to this in Python, so you 
have to do the steps yourself.

- the constructor takes an additional argument, the 'outer' object, 
which has to be kept in the object:
def __init__ (self, outer, ...):
 self.outer = outer

- when creating the inner object, the outer object must be passed to the 
  constructor
 obj = InnerClass (self)

- the outer object must be explicitely referenced:
 self.outer.increase (20)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Use cases for del

[Daniel Dittmar]

Peter Hansen wrote:
> Arguing the case for del: how would I, in doing automated testing, 
> ensure that I've returned everything to a "clean" starting point in all 
> cases if I can't delete variables?  Sometimes a global is the simplest 
> way to do something... how do I delete a global if not with "del"?

globals ().__delitem__ (varname)

except that the method would probably be called delete.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Modules for inclusion in standard library?

[Daniel Dittmar]

Rocco Moretti wrote:
>>> Except that (please correct me if I'm wrong) there is somewhat of a
>>> policy for not including interface code for third party programs which
>>> are not part of the operating system. (I.e. the modules in the
>>> standard libary should all be usable for anyone with a default OS +
>>> Python install.)

There are several degrees of std-ness for Python Modules:

Example expat: The sources for this extension module are contained in 
the Python tarball, so this module is guaranteed to be part of a Python 
distribution

Example zlib: If the include files and libs can be found, this extension 
will be built. So it's part of std-Python (the source), but not part of 
std-Python (installed). See others in Modules/Setup

Binary distributions may choose additional modules which appear to be 
standard. Example: zlib is part of all Python-Installations on Windows 
and most Linux-distribution will have readlines as 'standard'.

There seems to be a great reluctance by the Python developers to add 
modules of the expat kind, as this means responsibilities for additional 
source modules. There's also the problem with incompatible licenses, 
integrating a second configure, deciding when to update to the latest 
version of the library etc.

Another problem is that there are often several modules for a specific 
problem (e.g. several modules interfacing to PostgreSQL), so there's 
always a chance to make an anemy for life, no matter which you pick.

And module authors might not be interested as they'd be then bound by 
the Python release cycle.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about Python

[Daniel Dittmar]

Jan Danielsson wrote:
>But then it occured to me.. I started writing my program in Java
> pre-1.5. Then came 1.5, I upgraded, and my program would still compile
> and run, though I did get three warnings. The language had changed a
> little bit; I had to assign a type to three arrays. That wasn't so bad.
> 
>However, when I look at the various Python modules/libraries, I see
> that there are several versions of them, for different versions of
> python. I've seen everything from "for python 1.5" up to "for python
> 2.4" with all versions in between. This scares me a little bit. I assume
> that the reason for the different versions is because of new language
> features?

Python is generally backwards compatible

* pure Python modules will work unchanged when used with a newer version 
of Python (so it's almost always: for Python 1.5 and later). You might 
get some deprecation warnings and these appear one every run (compared 
to Java, where they appear only when compiling). But they can be disabled.

* Python extension modules written in C can mostly be recompiled from 
source with the new headers  and will work as well

* binary extensions generally won't work with newer Python, that's why 
you often see several downloads for a specific module. On Unix, binary 
compatibility worked pretty well up to 2.2 (Meaning I could use  modules 
compiled for 1.5 with Python 2.2), but that was just a coincidence and I 
wouldn't rely on it for future versions.

Summary: If you are able to compile all your needed extensions yourself, 
then new Python versions aren't really a problem.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python syntax high-lighting and preservation on web

[Daniel Dittmar]

Gregory Piñero wrote:
> Hey guys,
> 
> Does anyone know where I can pick up a style sheet (css) and/or other
> files/programs I might need to display python code on my website with
> tab preservation(or replace with spaces) and colored syntax?  I want
> something similar to the python code on a page like this:

see http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52298

css only won't help you as someone has to insert the span tags so that 
the style sheet has something to match.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Which kid's beginners programming - Python or Forth?

[Daniel Dittmar]

[EMAIL PROTECTED] wrote:
> List comprehensions, however, *are* the basic control flow; loops are
> much more verbose and they should be used only when necessary.

List comprehensions are probably a bad idea for entry level programmers:
- for and while loops are much easier to debug as you can insert print 
statements everywhere
- list comprehensions don't allow you to break complex expressions into 
several simpler ones by using local variables, everything has to happen 
in one expression

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Which kid's beginners programming - Python or Forth?

[Daniel Dittmar]

BORT wrote:
> I told my son, who wants to learn how to compute probabilities, that we
> have to start with some boring stuff so we can learn how to do the cool
> stuff.  Adding and subtracting aren't really fun, but figuring odds on
> rolling dice IS fun.  Learning to program will be kind of like that.
> He accepted that explantion.

I'm not sure that you actually have to start with the boring stuff. 
Imagine that you have a small, but complete program that executes some 
random function a thousand times and plots the distribution. Your son 
could probably
* start to change parameters to the function
* try out the different distributions in the library
* combine them to form new distributions (e.g. roll two n-sided dice)
* build more complex simulations (pit two Dungeons&Dragons fighters 
against each other by rolling simulated dice)

It's a bit more work for you as you'll have to decide on each step how 
much of the infrastructure you implement without taking away all the 
challenges.

Python vs. FORTH: what you learn from Python is more easily transferred 
to other programming languages. And if you happend to speak German, 
there is "Python für Kids" 


Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP ? os.listdir enhancement

[Daniel Dittmar]

Riccardo Galli wrote:
> On Thu, 23 Jun 2005 11:34:02 +0200, Andreas Kostyrka wrote:
> 
> 
>>What's wrong with
>>
>>(os.path.join(d, x) for x in os.listdir(d))
>>
>>It's short, and easier to understand then some obscure option ;)
>>
>>Andreas
> 
> 
> how does it help in using list comprehension, as the ones in the first
> post?

You can nest list comprehension
[ e
 for e in (os.path.join(d, x) for x in os.listdir(d))
 if os.path.isdir (e)]

You might also want to look at module itertools, which has better 
support for transforming and filtering in multiple steps.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP ? os.listdir enhancement

[Daniel Dittmar]

Riccardo Galli wrote:
> On Thu, 23 Jun 2005 12:56:08 +0300, Konstantin Veretennicov wrote:
>>What about os.listdir(dir='relative/path', abs=True)? Should listdir call
>>abspath on results? Should we add another keyword rel? Would it complicate
>>listdir unnecessarily?
> 
> keyword dir not exists (don't know if you added as example or not) and
> abs set to true would return abspath on result. What else could it do ?

He probably meant that a 'join' option would be more natural than an 
'abs' option. After all, your examples use os.path.join to create a 
valid path that can be used as the argument to other module os 
functions. Whether the results are absolute or relative should depend on 
the initial argument to os.listdir.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: *Python* Power Tools

[Daniel Dittmar]

Micah wrote:
> Anyone know if there is any organized effort underway to implement the
> Python equivalent of "Perl Power Tools" ?
> 
> If not, would starting this be a waste of effort since:
> 
> - it's already being done in Perl?
> - cygwin thrives?
> - UNIX is already pervasive :-) ?
> 
> Or would people really like to claim a pure Python set of UNIX
> utilities?

There would be some use for a Python library that implements the 
functionality of these tools. Some of it already exists (difflib, 
shutil). Some is probably more difficult and less flexible to use from a 
lib that from straight python (cut).

Perhaps that could be started either in the Python Cookbook 
(http://aspn.activestate.com/ASPN/Python/Cookbook/) or the Python Wiki 
(http://wiki.python.org/moin/). Just short explanations of how you would 
implement the functionality of various Unix utilities in Python.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple instances of a python program

[Daniel Dittmar]

Rahul wrote:
> Hi.
> I am part of a group in my univ where we organize a programming
> contest. In this contest we have a UDP based server. The server
> simulates a game and each contestant is to develop a team of virtual
> players. Each team is composed of 75 similar bots...i.e. governed by
> the same logic. Thus the contestant submits a single copy of the client
> and we instantiate the same program 75 times at the same time.
> The problem is that while executables from C source files are small and
> we can make 75 processes but we dont know what to do with python.
> 
> If you have a python script and you want that 75 copies of the script
> be run simultaneously how will you do it? Is there anyway to do so
> without running 75 copies of the python interpreter simultaneously?
> 

The technical way would be to use threads.

Of course it could be that the rules of the game explicitly forbid that 
bots of a group communicate other than through the server. And it is 
easier to cheat there when you have only one Python program running.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Which Python Wiki engine?

[Daniel Dittmar]

Kenneth McDonald wrote:

> I'm looking for a Wiki engine to set up for my company, so that we  can 
> incrementally add user documentation for a fairly complex  program, plus 
> allow users to add their own comments for the benefit  of others. I'd 
> strongly prefer a Python-based Wiki, since that allows  me the chance to 
> add plugins with minimal effort (being a python  enthusiast and 
> programmer).

ZWiki has support for showing an outline of the pages. But other than 
that, I found it less useful than MoinMoin. Also, ZWiki is a Zope 
product, and to me, Zope programming is not really Python programming.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MySQLDBAPI

[Daniel Dittmar]

Gregory Piñero wrote:
> building '_mysql' extension
> gcc -pthread -shared build/temp.linux-i686-2.4/_mysql.o -lz
> -lmysqlclient_r -o build/lib.linux-i686-2.4/_mysql.so
> /usr/bin/ld: cannot find -lmysqlclient_r
> collect2: ld returned 1 exit status
> error: command 'gcc' failed with exit status 1

Now search for mysqlclient_r.so or mysqlclient_r.a (probably in 
rpmmysql/usr/lib.

A mysqlclient.so (without _r meaning without support for threading) will 
do as well if you use Python as a CGI program.

Now enter the directory where you found mysqlclient to setup.py
library_dirs = ['']

If you found both .so and .a, rm the .so. The linker will then link the 
static library .a and you can copy the resulting _mysql.so everywhere.

If there is only an .so, be sure to copy the mysqlclient*.so to the 
server together with _mysql.so. You probably have to inspect the 
environment in your CGI run for LD_LIBRARY_PATH for a directory where 
you can copy mysqlclient*.so. (That's why it is easier to use the static 
library)

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MySQLDBAPI

[Daniel Dittmar]

Gregory Piñero wrote:
> Is that something I can install all to my home directory?

If you have a similar Linux distribution at home, simply build the mysql 
extension on that machine and then copy it to the web server.

Otherwise:

You don't have to actually install it. Just make sure that Setup.py 
finds the headers and libs

1a) get the rpm and extract the files

or 1b) compile (but don't install) mysql from sources

2) specify the locations of the header files and the libs to Setup.py or 
patch the module such that the defaults point to your directories. From 
your error message, the current default for the includes seems to be 
/usr/local/mysql/include/mysql

3) You can then delete the directories created in 1a or 1b

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lists in cx_Oracle

[Daniel Dittmar]

Andrew Dalke wrote:
> It sounds like you're saying that the interface is actually implemented
> by passing the execute string and a database-specific dictionary-like
> object; the latter created by the DB-API interface.

That's the way it's supposed to work. The program prepares a statement 
with placeholders only once and sends it with varying parameters to the 
database. That way, the statement has to be parsed by the database only 
once and the execution plan can be reused (with Oracle even between 
different sessions). It seems as if nothing ticks Oracle DBAs more off 
than if you're complaining about poor performance, but you're not using 
these prepared statements.


The :placeholder syntax comes from embedded SQL. You would write the 
names of actual program variables there and a precompiler would generate 
the code to 'bind' the program variables to SQL parameters. Embedded SQL 
has fallen out of favour and new APIs (ODBC, JDBC) use the question mark 
as the placeholder. The principle remains the same for many databases: 
the SQL string is sent unchanged to the database. Additionally, a list 
of actual values is sent for each execution.


Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lists in cx_Oracle

[Daniel Dittmar]

Andrew Dalke wrote:
> I want to execute a query with an "IN" in the WHERE clause
> and with the parameter taken from a Python variable.  That
> is, I wanted something like this to work
> 
> id_list = ["AB001", "AB002", "AB003"]
> 
> c.execute("""SELECT s.smiles FROM smiles_database s WHERE """
>  """  s.id IN :id_list""", id_list = id_list)
> 
> I couldn't get it to work.  It complained
> 
> arrays can only be bound to PL/SQL statements
> 

Possible workarounds:
- use executemany: a few databases allow to execute several sets of 
input parameters at once. And even fewer allow this for SELECTs, where 
the result is one cursor created from the UNION of a SELECt for each set 
of input parameters. Apart from being unlikely to work, this also 
requires that *all* input parameters are lists of the same length 
(unless the driver is smart enough to expand skalars to lists in this 
context)

- specify a maximum number of input parameters 's.id in (:id0, :id1, 
...)' and fill missing values with the first value

- create a class for this purpose. Statement are created on the fly, but 
with placeholders so you don't run into the SQL Injection problem. As 
it's an object, you could cache these generated statements base on the 
size of the list

- create a temporary table, insert all the values into that table 
(executemany is great for INSERTS) and then join with that table

You could also search comp.language.java.database where this is a 
frequent question.

It is unlikely that this can be solved at the driver level. Without 
support from the database, the driver would have to manipulate the SQL 
statement. And there are few predicates where a list parameter is 
useful. Expanding a list always yould lead to very bizarre error 
messages. Expanding them only where useful would require a SQL parser.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Getting into Python, comming from Perl.

[Daniel Dittmar]

Miguel Manso wrote:
I've tryed to use python some times but I get frustrated very quick. I 
get myself many times needing to figure out how to loop through a list, 
declare an associative array, checking how to pass named parameters to 
functions, and simple things like that.
Create a cheat sheet with all those stock phrases that you know by heart 
in Perl, but have to look up in Python. That way, it's only one look. 
And you'll know them by heart soon enough.

You could also print out or keep bookmarks on the modules you use the 
most. When I started using Python, I did this for strings, regular 
expressions and the os and os.path modules.

Daniel
--
http://mail.python.org/mailman/listinfo/python-list

Re: Case-insensitive dict, non-destructive, fast, anyone?

[Daniel Dittmar]

Ville Vainio wrote:
"Daniel" == Daniel Dittmar <[EMAIL PROTECTED]> writes:

Daniel> Ville Vainio wrote:
>> I need a dict (well, it would be optimal anyway) class that
>> stores the keys as strings without coercing the case to upper
>> or lower, but still provides fast lookup (i.e. uses hash
>> table).
Daniel> Store the original key together with the value and use a
Daniel> lowercase key for lookup.
That's what I thought initially, but the strings take most of the
space in dict and I didn't feel like doubling the size.
You could write a string wrapper that changes comparison and hashing. 
I'm not sure that this (+ 1 (Python object + instance dictionary)) would 
use less memory than the other proposal (+ 1 Tuple + 1 String).

Daniel
--
http://mail.python.org/mailman/listinfo/python-list

Re: Case-insensitive dict, non-destructive, fast, anyone?

[Daniel Dittmar]

Ville Vainio wrote:
I need a dict (well, it would be optimal anyway) class that stores the
keys as strings without coercing the case to upper or lower, but still
provides fast lookup (i.e. uses hash table).
Store the original key together with the value and use a lowercase key 
for lookup.

only a sketch:
class MyDict:
def __init__ (self):
self.inner = {}
def __setitem__ (self, key, value):
self.inner [key.lower ()] = (key, value]
def __getitem__ (self, key):
realkey, realvalue = self.inner [self]
return realvalue
def get (self, key, default = None):
try:
return self [key]
except KeyError:
return default
# or: return self.inner.get (key.lower (), (None, default)) [1]
def keys (self):
return [realkey for realkey, realval in self.inner.values ()]
def values (self):
return [realval for realkey, realval in self.inner.values  )]
   def items ():
return self.inner.values ()
# iterators are left as an exercise
--
http://mail.python.org/mailman/listinfo/python-list

Re: Optimisation Hints (dict processing and strings)

[Daniel Dittmar]

OPQ wrote:
- Try if it isn't faster to iterate using items instead of iterating 
over keys

items are huge lists of numbers. keys are simple small strings. And
even if it is faster, how can I find the key back, in order to delete
it ?
for v in hashh.items():
if len(v)<2:
   del ???
To elaborate on the memory requirements of .keys () vs. items ():
.keys () creates a new list of n objects. The objects are additional 
references to the existing keys.

.items () creates also a new list of n objects. These objects are tuples 
 of references, one to the key and one to the value. Only references 
are used so it doesn't  matter how large the value actually is. Whether 
the tuples are created for the items () call or already exist depends on 
the implementation of the dictionary. Trying to infer this by using 
sys.getrefcount got me inconclusive results.

I gonna try, but think that would be overkill: a whole list has to be
computed !
Maybe whith genexps .. for key in (k for (k,v) in hash.iteritems()
if len(v)<2)
Using only iterators has problems:
for k,v in hash.iteritems ():
if len (v) < 2:
del hash [k]
You are changing hash while you iterate over it, this very often breaks 
the iterator.

If you are memory bound, maybe a dabase like SQLite is really the way to 
go. Or you could write the keys to a remporary file in the loop and then 
write a second loop that reads the keys and deletes them from hash.

Daniel
--
http://mail.python.org/mailman/listinfo/python-list

Re: Optimisation Hints (dict processing and strings)

[Daniel Dittmar]

OPQ wrote:
for (1): 

longone=longone + char # where len(char)== 1
I known that string concatenation is time consuming, but a small test
on timeit seems to show that packing and creating an array for those 2
elements is equally time consuming
- use cStringIO instead
- or append all chars to a list and do "".join (listvar)
for (2):
for k in hash.keys()[:]: # Note : Their may be a lot of keys here
   if len(hash[k])<2:
  del hash[k]
Here again, I think the hash.keys duplication can be time *and* memory
consuming. But still better than (I suppose - no time it yet)
hash=dict([(k,v) for (k,v) in hash if len(v)>1])
- Try if it isn't faster to iterate using items instead of iterating 
over keys
- use the dict.iter* methods to prevent building a list in memory. You 
shouldn't use these values directly to delete the entry as this could 
break the iterator:

for key in [k for (k, v) in hash.iteritems () if len (v) < 2]:
del hash (key)
This of course builds a list of keys to delete, which could also be large.
- also: hash.keys()[:] is not necessary, hash.keys () is already a copy
Daniel
--
http://mail.python.org/mailman/listinfo/python-list

Re: The Running Time of += on Char Strings ?

[Daniel Dittmar]

Edg Bamyasi wrote:
What is the running time of conactination on character strings.
i.e.

joe="123"
joe+="9"

is it Amortized Constant time? I don't think it would be O((number of
chars)^2) but i really don't know.
Strings are immutable, so
joe+="9"
is executed as
joe = joe + "9"
This means that there is
- one allocation
- two memcpy
- one deallocation (old value of joe)
My guess is that the allocations play a rather large part in the actual 
timing.

Creating a large string by multiple concatenations is slow, instead you 
should:
- use module cStringIO
- or add all the strings to a list and do "".join (listvariable)

How are you supposed to know? It's mostly Python folklore, some of which 
has been written down in the Python Cookbook 
(http://aspn.activestate.com/ASPN/Python/Cookbook/)

Daniel
--
http://mail.python.org/mailman/listinfo/python-list