Re: Some C-API functions clear the error indicator?

[Austin Bingham]

That makes a lot of sense. And if I take the approach that any Py*
function might do this, it actually looks like I can simplify my code
(rather than managing some list of ill-behaved functions or
something.) Thanks!

On Fri, Jan 29, 2010 at 3:58 PM, Duncan Booth
 wrote:
> Austin Bingham  wrote:
>
>> The functions that do this don't seem to indicate in their
>> documentation that this will happen. So first, does anyone know why
>> this is happening? Is it because of the context in which I'm making
>> the calls? Is there any pattern or reason behind which functions will
>> do this? Or am I just doing something wrong?
>>
> (Just guessing here)
> I would expect that any function that executes Python code will clear the
> error.
>
> I think that has to happen otherwise the Python code will throw an
> exception whenever it gets round to checking for errors. In the past I've
> found that if you fail to check for an error in C code before returning to
> the interpreter you get the exception thrown a few instructions later, so
> something similar would probably happen if you call other Python code from
> C.
>
> If it is anything that executes Python then that would include any function
> that creates or destroys an object with Python constructor or destructor
> code. or that compares or otherwise operates on instances defined in
> Python. In particular it might mean that any function that doesn't appear
> to clear the error could do so in a slightly different situation.
>
> --
> Duncan Booth http://kupuguy.blogspot.com
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Some C-API functions clear the error indicator?

[Austin Bingham]

The original post was, in a nutshell, the "use case"; it's very scaled
down the from the real example, and not intended to "make sense". The
notion on which I was (apparently incorrectly) basing my exception
translation was that I could 1) get and reset references to the error
indicators, 2) call other python methods that don't themselves throw,
and 3) later retrieve the same "active" exceptions. I was relying on
this ability to "re-fetch" exceptions insofar as the functions in my
code which dealt with python exceptions all looked up the exception
separately. The predicate that "a successful function won't modify the
error indicators" appears to be wrong, however, and I've modified my
code accordingly.

Austin

On Sat, Jan 30, 2010 at 1:11 AM, Gabriel Genellina
 wrote:
> En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham
>  escribió:
>
>> Maybe I'm not following what you're saying. In my case, I already know
>> that an exception has been thrown. In the course of processing that
>> exception, I call another function which, for whatever reason and even
>> when it succeeds, clears the exception indicators. How can I address
>> this issue by checking function calls for failure?
>
> Maybe if you provide an actual use case we can suggest how to handle it. The
> code in your original post does not make any sense to me (except by showing
> that PyImport_ImportModule does clear the error indicator). If you already
> know there was an error, and you even have retrieved the error details, why
> do you care if the error indicator gets reset?
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C API: module cleanup function

[Stefan Behnel]

Mr.M, 29.01.2010 23:50:
> I can't figure out if there is a way to run a specialized cleanup
> function when a module needs to be "unloaded" (i.e. just before a
> reload() or when i quit the interpreter).
> 
> I'm thinking of something like tp_dealloc.
> 
> If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find
> that Python provides a default tp_dealloc for me.
> 
> Now, suppose my module needs to allocate some resources at startup, I'm
> not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right?
> 
> But, if I reload() my module or if I quit the Python interpreter, I'd
> like to free those resources (before allocate them again, in case of a
> reload).
> 
> Is there a way to make this work?

Gabriel already pointed you to the module cleanup support in Py3, which can
be used to provide reload capabilities to your module.

In Py2, there are at least some ways to free resources when terminating the
interpreter. See the "atexit" module and the Py_AtExit() function:

http://docs.python.org/library/atexit.html
http://docs.python.org/c-api/sys.html

Note that both have their specific limitations, though, as you can see from
the docs.

Also note that it might help you to take a look at Cython, a Python-to-C
compiler for writing fast C extensions. It has an option for generating
module-level cleanup code automatically, and generally simplifies writing
binary extension modules quite a bit.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Code snippet: dualmethod descriptor

[Steven D'Aprano]

I came up with this descriptor a few years ago, but never used it for 
anything. I've just found an actual use for it in my own code, so I 
thought I'd make it public for anyone else who might have a use for it.

Use-case: if you have a class without an __init__ or __new__ method, then 
this may be of use to you. It enables the caller to call methods on 
either the class, or an instance, and the first argument passed the 
method will be the class or the instance respectively.

This differs from classmethods, which always passes the class, and 
staticmethods, which don't pass either.

The name "dualmethod" is my own.

Here's the implementation:

class dualmethod(object):
"""Descriptor implementing dualmethods (combination 
class/instance method).

Returns a method which takes either an instance or a class as 
the first argument. When called on an instance, the instance is
passed as the first argument. When called as a class, the class
itself is passed instead.

>>> class Example(object):
... @dualmethod
... def method(this):
... if type(this) is type:
... print("I am the class '%s'." % this.__name__)
... else:
... print("I am an instance of the class '%s'." %
... this.__class__.__name__)
...
>>> Example.method()
I am the class 'Example'.
>>> Example().method()
I am an instance of the class 'Example'.

"""
def __init__(self, func):
self.func = func
def __get__(self, obj, cls=None):
if obj is None: obj = cls or type(obj)
def newfunc(*args, **kwargs):
return self.func(obj, *args, **kwargs)
return newfunc


(Tested with Python 2.4, 2.5, 2.6, 3.0 and 3.1.)


My use-case is a class that has constants in class attributes, and 
methods which refer to those constants. Because they are constants, there 
is no reason to give each instance a separate reference to the attribute, 
hence the attribute is on the class:

class Example:
ATTR = "something"
@dualmethod
def method(this):
return this.ATTR + " else"

Normally I have no reason to instantiate the class: there is no __init__ 
or __new__ method. Normally I would use classmethod, but every now and 
again I want to change the behaviour by modifying ATTR, and using class-
methods would force me to subclass in order to make that change.

dualmethod lets me get the best of both worlds. If I'm using the default 
value of ATTR, I can just call the method on the class without bothering 
to instantiate it. But if I want to customise the behaviour of the 
method, rather than subclassing, I can instantiate and add an instance 
attribute:

>>> Example.method()
'something else'
>>> x = Example()
>>> x.ATTR = "nothing"
>>> x.method()
'nothing else'
>>> Example.method()  # default still works as expected
'something else'


without effecting the default behaviour.

I hope this is useful to you. You can use this without condition, 
although I would be grateful for credit.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Ben Finney]

Blog  writes:

> (Debian does ship with 2.5, but the next major release "sid' is due
> out in Q2)

Sid is the perpetual development playground (“unstable”), never released
as a suite, but a proving ground for packages to determine their fitness
for going to the next level of testing.

The next-to-be-released suite is Squeeze (currently “testing”), which
has Python 2.5 (the default ‘python’) and Python 2.6.

-- 
 \   “If you define cowardice as running away at the first sign of |
  `\   danger, screaming and tripping and begging for mercy, then yes, |
_o__)   Mr. Brave man, I guess I'm a coward.” —Jack Handey |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Blog]

On 1/28/2010 8:44 AM, Paul Rubin wrote:

Steve Holden  writes:

Kindly confine your debate to the facts and keep the snide remarks to
yourself. Like it or not Python 3 is the future, and unladen swallow's
recent announcement that they would target only Python 3 represented a
ground-breaking advance for the language.


My take on things is that doing unladen swallow really "right" will
require yet more incompatible changes; i.e., the result will either
still leave quite a bit of performance on the table, or else it won't be
compatible with the current specification of Python 3 and they'll
presumably have to call it Python 4.  And if Python 4 is as good as I
believe it could possibly be, then it might get wide acceptance before
Python 3 really has all that much uptake.  If I have to accept
incompatibility anyway, and Python 4 gives huge improvements while
Python 3's improvements are tiny or moderate, why not skip over Python 3?


There's a prime example - it's called Windows Vista! ;)

--
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Blog]

On 1/28/2010 2:56 AM, John Nagle wrote:

Daniel Fetchinson wrote:
1. Python 3 is supported by major Linux distributions.

FALSE - most distros are shipping with Python 2.4, or 2.5 at best.


Where did you come up with that information? Almost all of the major 
distros ship with 2.6.x - CentOS, OpenSuSe, Ubuntu, Fedora. (Debian does 
ship with 2.5, but the next major release "sid' is due out in Q2)



Arguably, Python 3 has been rejected by the market. Instead, there's
now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into
a debacle like Perl 6, now 10 years old.


WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE 
LAST version of the 2.x series - "the" End-Of-Life for Python 2.


At least py3k is a real product - unlike the perl 6 vaporware.

That said, I think python 2 will be the dominant player in the market 
for the next couple of years.


However, there's been some momentum behind Py3k recently. Unladen 
swallow is going to be merged with Py3k (not 2.x for which it was 
originally developed). I hear the django guys have made huge progress in 
porting the framework to Py3k (although it will be a good few months 
before the code is released in the wild)


--
http://mail.python.org/mailman/listinfo/python-list

Re: Threading issue with SQLite

[Stephen Hansen]

On Fri, Jan 29, 2010 at 8:37 AM, Alan Harris-Reid <
aharrisr...@googlemail.com> wrote:

> Hi,
>
> I am creating a web application (using Python 3.1 and CherryPy 3.2) where a
> SQLite connection and cursor object are created using the following code
> (simplified from the original):
>
> class MainSite:
>   con = sqlite.connect('MyDatabase.db')
>   cursor = con.cursor()
>

This is bad. For one thing, every thread should have its own connection at
least-- you can store it in thread-local storage (see the threading module)
for re-use if you want, if cherrypy does thread pooling techniques or some
such (I'd be sorta surprised if it didn't).

For another thing, cursors are NOT meant to be long-running interfaces.  You
should not leave one open beyond one concrete action (e.g., one user
request, with however many SQL statements it takes to accomplish that work)
even if you are re-using the same connection across multiple
methods/invocations.

Questions...
> 1.  Is there a large overhead in opening a new SQLite connection for each
> thread (ie. within each method)?
>

You're making a web-app in Python, man. :) The overhead of a SQLite
connection is not exactly significant at that point :) Though you could use
thread pooling techniques with each thread having its own dedicated
connection to mitigate what overhead there is.


> 2.  Is there any way to use the same connection for the whole class (or
> should I forget that idea completely?)
>

You shouldn't think of "connection" and "class" as having any kind of
meaningful link, it just doesn't make sense. It won't work and what you're
-trying- to do doesn't actually even make sense. Besides the fact that
you're worrying prematurely about overhead and trying to optimize it away,
connections (and from them, cursors) to databases are things that are used
for binding together a bunch of statements into transactions and maintaining
'current state' of your interaction with the database: its a a logical
abstraction of "what I'm doing Now to the database", whereas the controller
class is, "what can be done."

3.  When a method returns to the calling method, is the connection
> automatically closed (assuming the object is local, of course) or does it
> have to be done explicitly using connection.close()?
>

If it goes out of scope, it will be closed if its a local variable (be sure
you commit if you need to first!). Assuming you aren't caching/saving the
connection in the thread for future calls it handles, if you have a thread
pool doing your work (again, I'd be -really- surprised if CherryPy with
threads didn't involve thread pools as opposed to starting up and stopping
threads continually as requests come in).

HTH,

--S


>
> TIA,
> Alan Harris-Reid
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


--S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: For loop searching takes too long!

[Steven D'Aprano]

On Fri, 29 Jan 2010 14:49:06 -0800, Jonathan Gardner wrote:

> On Jan 28, 3:52 pm, elsa  wrote:
>>
>> I've got a problem with my program, in that the code just takes too
>> long to run. Here's what I'm doing. If anyone has any tips, they'd be
>> much appreciated!
>>
>>
> First of all, don't play with large lists. Large lists have a tendency
> to grow larger over time, until they get absurdly large.
> 
> If you're dealing with a long list, work with it like you'd work with a
> file. If you need random access, stick it in some form of database, such
> as BDB or PostgreSQL. If you need an index, stick it in some form of DB.
> Eventually, large lists end up like that anyway. Don't fool yourself
> early on---prepare for the worst and solve the problems of tomorrow
> today.


Talk about premature optimization. The OP probably has a few hundreds of 
thousands of items, millions at most, which is trivial on a modern PC. 
Your way of thinking is what has led to Firefox using a database to 
manage a few thousand bookmarks and cookies, which in turn leads to 
consistent data corruption problems. (I've been keeping note, and so far 
my installation of Firefox 3 has had database corruption at startup one 
time out of five.)



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Perl 6 [was Re: myths about python 3]

[Carl Banks]

On Jan 28, 9:34 pm, Steven D'Aprano  wrote:
> On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote:
> > Perl 6, on the other hand, is still fantasyware a decade after its
> > announcement.  It is, for the most part, THE canonical example of the
> > wrong way to conduct a development effort.
>
> Out of curiosity, and completely off-topic, why has Perl 6 gone so badly?

Because Larry Wall saw the writing on the wall that his "nice little
scripting language" was way out of its league in terms of what it was
trying to be (i.e., a real programming language suitable for building
applications), and so he bailed out and left the Perl community with
no leadership.

This is just my impression, but some of the efforts to create Perl 6
seem to be clinging to a notion that Perl has always been a good
language and are trying to ensure that it doesn't stray very far from
Perl's core principles.  Problem is, Perl wasn't ever a good language
so they won't succeed.  IOW, as Geremy Condra said, "it's too much
like Perl".


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Keyboard input

[Steven D'Aprano]

On Fri, 29 Jan 2010 18:38:47 -0300, Gabriel Genellina wrote:

>> I'm using the raw_input method and it works fine, but I noted that I
>> can't use backspace and when I accidentally press shift space (because
>> I need to input uppercase letters separated by a space) it writes some
>> strange characters.
> 
> That's strange. If you're using Linux, make sure you have the readline
> package installed.

Yes, I have this same issue too, I've just recently installed Python 2.0 
(as well as 1.5, 2.4, 2.5, 2.6, 3.0 and 3.1), and while all the other 
versions recognise the readline package, this doesn't:

>>> x = 1
>>> # hit up arrow to get previous line
... ^[[A


I'm pretty sure I built 2.0 the same way I built all the others, anyone 
have any hints as to what I missed?


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Benjamin Kaplan]

On Fri, Jan 29, 2010 at 5:29 PM, Ben Finney  wrote:
> Duncan Booth  writes:
>
>> Here's what I see in the Ubuntu packages. Python 3 seems only to be in the
>> universe repositories so far.
>>
>> Dapper: Python 2.4.2
>> Hardy: Python 2.5.2
>> Intrepid: Python 2.5.2, 3.0~b3 (universe)
>> Jaunty: Python 2.6.2, 3.0.1 (universe)
>> Karmic: Python 2.6.4rc1, 3.1 (universe)
>> Lucid: Python 2.6.4, 3.1 (universe)
>>
>> WTF is rc anything doing in the main repository?
>
> It's to be expected if the release team process specifies announcement
> messages of the form “RELEASED: Python X.Y.Zrc2”. I have long argued,
> without much traction, that the process should reserve the term
> “RELEASED” for use only in reference to releases, not other things.
>

Don't worry, it's not just Python. Ubuntu Hardy (IIRC) had a Firefox 3
beta in the repositories until a little while after the final release.
I don't know what's taking them so long to upgrade to the final 2.6.4
though.

> --
>  \     “First they came for the verbs, and I said nothing, for verbing |
>  `\    weirds language. Then, they arrival for the nouns and I speech |
> _o__)                           nothing, for I no verbs.” —Peter Ellis |
> Ben Finney
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: image processing - inverse filtering

[Aahz]

In article <5bfefbb6-89a8-49f6-9f02-7d36dfbc0...@c29g2000yqd.googlegroups.com>,
suresh.amritapuri  wrote:
>
>If I am using scipy.ndimage.gaussian_filter() for filtering an image,
>how to do the inverse filtering? In general how to do this using
>scipy.ndimage?

http://projects.scipy.org/mailman/listinfo/scipy-user
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

import antigravity
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Changing tab behavior in Python Interactive mode

[Terry Reedy]

On 1/29/2010 4:43 PM, Andrej Mitrovic wrote:

On Jan 29, 10:07 pm, Steve Holden  wrote:

Andrej Mitrovic wrote:

On Jan 29, 6:47 pm, Steve Holden  wrote:

Andrej Mitrovic wrote:

I've noticed that when running Python in interactive mode (via cmd on
windows), the first time I hit tab it will move 4 spaces to the right,



I's nothing to do with the source: it isn't the interpreter that's
deciding where the "tab stops" are, it's the console window the
interpreter is running in.



I see. Well I'm nitpicking here, it's not all that important (I just
use cmd when I'm trying out small pieces of code). And I doubt there's
a tabstop setting for cmd that I could use.


Nope. I checked the properties dialog (rt click upper left icon).
You might want to turn on quickeditmode (as admin).

Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Some C-API functions clear the error indicator?

[Gabriel Genellina]

En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham  
 escribió:



Maybe I'm not following what you're saying. In my case, I already know
that an exception has been thrown. In the course of processing that
exception, I call another function which, for whatever reason and even
when it succeeds, clears the exception indicators. How can I address
this issue by checking function calls for failure?


Maybe if you provide an actual use case we can suggest how to handle it.  
The code in your original post does not make any sense to me (except by  
showing that PyImport_ImportModule does clear the error indicator). If you  
already know there was an error, and you even have retrieved the error  
details, why do you care if the error indicator gets reset?


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 76, Issue 303

[Gabriel Genellina]

[top posting corrected]


From: Steve Holden 
Date: Tue, 26 Jan 2010 11:54:23 -0500



Anyway, I suspect your error might go away if you turned the first
argument of hte log.info() call into a format string such as

 log.info("refer: %s", ret)


try:
   log.info("start")
   log.info("refer",ret)
   log.info("end")
except TypeError:
   log.exception("Exception raised")


En Fri, 29 Jan 2010 06:17:56 -0300, siddhartha veedaluru  
 escribió:



Thats the output i got in output window.it is not listing the statement
which caused it. copy paste and try it on you setup. it behaves the same
way.

I know that error is in log.info() function call.but it is not caught by
"except". which is not getting logged using log.exception


Errors that occur inside the logging system are not logged themselves  
(users aren't usually interested on those kind of errors, but on  
application errors; also, if logging errors were logged too, this could  
create an infinite loop).


To handle those errors in the logging system yourself, override  
Handler.handleError()


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: C API: module cleanup function

[Mr.M]

Gabriel Genellina ha scritto:
I think what you want to do isn't possible with Python 2, and it's one 
of the reasons the module handling was redesigned in Python 3.x; see PEP 
3121.


Thank you Gabriel for your help. Unlucky I can't use Python 3.x in my 
project, sob!


Luca.
--
http://mail.python.org/mailman/listinfo/python-list

Re: C API: module cleanup function

[Gabriel Genellina]

En Fri, 29 Jan 2010 19:50:52 -0300, Mr.M  escribió:

I can't figure out if there is a way to run a specialized cleanup  
function when a module needs to be "unloaded" (i.e. just before a  
reload() or when i quit the interpreter).


I think what you want to do isn't possible with Python 2, and it's one of  
the reasons the module handling was redesigned in Python 3.x; see PEP 3121.



I'm thinking of something like tp_dealloc.


m_free (PyModuleDef member, in Python 3) might work for this.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

C API: module cleanup function

[Mr.M]

Hi,

I can't figure out if there is a way to run a specialized cleanup 
function when a module needs to be "unloaded" (i.e. just before a 
reload() or when i quit the interpreter).


I'm thinking of something like tp_dealloc.

If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find 
that Python provides a default tp_dealloc for me.


Now, suppose my module needs to allocate some resources at startup, I'm 
not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right?


But, if I reload() my module or if I quit the Python interpreter, I'd 
like to free those resources (before allocate them again, in case of a 
reload).


Is there a way to make this work?

Thank you, Luca.
--
http://mail.python.org/mailman/listinfo/python-list

Re: For loop searching takes too long!

[Jonathan Gardner]

On Jan 28, 3:52 pm, elsa  wrote:
>
> I've got a problem with my program, in that the code just takes too
> long to run. Here's what I'm doing. If anyone has any tips, they'd be
> much appreciated!
>

First of all, don't play with large lists. Large lists have a tendency
to grow larger over time, until they get absurdly large.

If you're dealing with a long list, work with it like you'd work with
a file. If you need random access, stick it in some form of database,
such as BDB or PostgreSQL. If you need an index, stick it in some form
of DB. Eventually, large lists end up like that anyway. Don't fool
yourself early on---prepare for the worst and solve the problems of
tomorrow today.

> So, say I have a list of lists that looks something like this (I'm
> using a list of lists, rather than a list of tuples, as I need it to
> be mutable):
>
> myList = [[786,0],[45, 1],[673,1],...[23,46]]
>
> there are enough entries in the outer list, that the sum of myList[i]
> [0] across all i could be as high as 10^7.
>
> Now, what I need to do is randomly choose one myList[i], however the
> distribution of my random choice needs to be proportional to the
> values of myList[i][0]. So, for this list above, I'd have a much
> higher chance of choosing myList[0] than myList[1].
>

Let's do some thinking here.

Let's say you have a list of N items. You need to choose one, but you
don't know how many there are.

One algorithm that works is you start with the first item. If that's
the only item, you've chosen it.

If there's another item, you roll the dice. There's a 1/2 chance you
drop the first item and take the sedon.

If there's a third item, you roll the dice, There's a 1/3 chance you
drop the first or second item, and take the third.

You see the pattern? Keep the nth item 1/n of the time.

In your case, you're actually pulling off N items each time, all with
the same value. Your chance of keeping the next item is (number of
items in the next one)/(total items seen so far, including the next
one). Do the math for this. It's really simple.

Let's walk through this:

[[786,0],[45, 1],[673,2],...[23,46]]

Step 1: See [786,0]. Remember 786. Keep 0.

Step 2: See [45,1].  Remember 786+45=831. Keep 1 45/831 of the time.

Step 3: See [673,2]. Remember 831+673=1504. Keep 2 673/1504 of the
time.

Etc...

Now, the algorithm I've just described is much less memory intensive
and can deal with very long lists of numbers. However, you are calling
rand() a lot. rand() is not a cheap operation. One optimization is to
roll rand() once, and keep using that number over and over again.
Granted, for very large numbers of items, you're going to need a very
precise random number of some very small probabilities will never be
chosen. This is left as an exercise for the reader.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Ben Finney]

Duncan Booth  writes:

> Here's what I see in the Ubuntu packages. Python 3 seems only to be in the 
> universe repositories so far.
>
> Dapper: Python 2.4.2
> Hardy: Python 2.5.2
> Intrepid: Python 2.5.2, 3.0~b3 (universe)
> Jaunty: Python 2.6.2, 3.0.1 (universe)
> Karmic: Python 2.6.4rc1, 3.1 (universe)
> Lucid: Python 2.6.4, 3.1 (universe)
>
> WTF is rc anything doing in the main repository?

It's to be expected if the release team process specifies announcement
messages of the form “RELEASED: Python X.Y.Zrc2”. I have long argued,
without much traction, that the process should reserve the term
“RELEASED” for use only in reference to releases, not other things.

-- 
 \ “First they came for the verbs, and I said nothing, for verbing |
  `\weirds language. Then, they arrival for the nouns and I speech |
_o__)   nothing, for I no verbs.” —Peter Ellis |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: which one is faster?

[Jonathan Gardner]

On Jan 28, 10:29 pm, "Stephen.Wu" <54wut...@gmail.com> wrote:
> str.find(targetStr)
> str.index(targetStr) with exception
> str.count(targetStr)
> targetStr in str
>
> which is the fastest way to check whether targetStr is in str?
>

The fastest way of all is to forget about this and finish the rest of
your program. Developer time is much, much more valuable than
processor time.

When you are all done, and have solved all the other problems in the
world, you can come back and pontificate on how many nanoseconds you
can save by using "find" or "in".

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Carl Banks]

On Jan 29, 12:25 am, "Martin v. Loewis"  wrote:
> > Well, I'd consider that an official release.  Note that I didn't claim
> > there was no hope PSF wouldn't change it's mind on 2.8.
>
> I'd like to point out that the PSF formally doesn't have any say in
> this.

Doesn't PSF own the Python trademark?  Then it has to have a say, not
over whether someone can fork the project or make another official
release, but over whether they can do so and still call it Python.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Threading issue with SQLite

[Jonathan Gardner]

On Jan 29, 8:37 am, Alan Harris-Reid 
wrote:
>
> Questions...
> 1.  Is there a large overhead in opening a new SQLite connection for
> each thread (ie. within each method)?

Yes, but not as bad as some other DBs.

> 2.  Is there any way to use the same connection for the whole class (or
> should I forget that idea completely?)

Forget it. The thread stuff is pretty important.

> 3.  When a method returns to the calling method, is the connection
> automatically closed (assuming the object is local, of course) or does
> it have to be done explicitly using connection.close()?
>

If the object falls out of scope, that is, no code anywhere can
reference it, it will be garbage collected and thus closed.

Suggestion: Use something like SQLAlchemy to manage you DB
interactions. One day, you'll move away from SQLite, and you'll be
glad you were programming at a higher level. SQLAlchemy also does
things like connection pooling and such for you.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Keyboard input

[Jonathan Gardner]

On Jan 29, 8:53 am, "Mr.SpOOn"  wrote:
> Hi,
> I need to get keyboard input in a python program. I need it to let the
> user choose some options, for example:
>
> 1) option 1
> 2) option 2
> 3) option 3
>
> and then to input some data to the program.
>
> I'm using the raw_input method and it works fine, but I noted that I
> can't use backspace and when I accidentally press shift space (because
> I need to input uppercase letters separated by a space) it writes some
> strange characters.
>
> So, is there another way to get keyboard input?
>
>

Maybe curses is overkill, but it will do what you need. Most
interactive console apps use curses. You've run into some of the
problems of simply reading STDIN.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Keyboard input

[rantingrick]

On Jan 29, 3:38 pm, "Gabriel Genellina" 
wrote:

> but it's a big  
> move. wxPython would let you write a graphical interface (and it's a  
> bigger move!)
>
> --
> Gabriel Genellina

*ahem*!, tkinter is just a slightly medium move, and my god it's in
the stdlib for crying out loud!!


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Data exchange between Delphi and Python (Win)

[Gabriel Genellina]

En Thu, 28 Jan 2010 05:07:00 -0300, Durumdara   
escribió:



I have an exotic db, with exotic drivers, and it have buggy  ODBC driver.

But I have native driver - under Delphi.

I need to access this DB under Pylons (or mod_python).
[...]
I think to COM/OLE, because it is accessable from all program, and I  
think

to DLL (but DLL have problematic parameterisation).


So, you can query the database in Delphi, and you want to be able to do  
that from Python.


Your two ideas are feasible: you may write a DLL (or a COM object) in  
Delphi, that takes a query string, executes it, and returns the resulting  
data. All in Delphi code, no Python involved. Keep the memory allocation  
in Delphi - allocate and return a plain PChar buffer, and include a free()  
function to deallocate it when the other side is done using it.


From Python, you may use ctypes [1] to call the DLL functions, or pywin32  
[2] to invoke the COM object. Both ways are easy enough.


[1] http://docs.python.org/library/ctypes.html
[2] http://sourceforge.net/projects/pywin32/

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Changing tab behavior in Python Interactive mode

[Andrej Mitrovic]

On Jan 29, 10:07 pm, Steve Holden  wrote:
> Andrej Mitrovic wrote:
> > On Jan 29, 6:47 pm, Steve Holden  wrote:
> >> Andrej Mitrovic wrote:
> >>> I've noticed that when running Python in interactive mode (via cmd on
> >>> windows), the first time I hit tab it will move 4 spaces to the right,
> >>> however each new tab will move 8 spaces instead of 4. Why this
> >>> inconsistent behavior? And how could I change this to be consistent
> >>> and always move only 4 spaces?
> >> The first tab you are starting four characters in due to the interpreter
> >> already having printed ">>> ".
>
> >> regards
> >>  Steve
> >> --
> >> Steve Holden           +1 571 484 6266   +1 800 494 3119
> >> PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
> >> Holden Web LLC                http://www.holdenweb.com/
> >> UPCOMING EVENTS:        http://holdenweb.eventbrite.com/
>
> > I see. I wonder if I could modify this behavior.. I guess I'll have to
> > take a look at the source. Thanks anyway!
>
> I's nothing to do with the source: it isn't the interpreter that's
> deciding where the "tab stops" are, it's the console window the
> interpreter is running in.
>
> regards
>  Steve
> --
> Steve Holden           +1 571 484 6266   +1 800 494 3119
> PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
> Holden Web LLC                http://www.holdenweb.com/
> UPCOMING EVENTS:        http://holdenweb.eventbrite.com/

I see. Well I'm nitpicking here, it's not all that important (I just
use cmd when I'm trying out small pieces of code). And I doubt there's
a tabstop setting for cmd that I could use.

Thanks for all your help!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Keyboard input

[Gabriel Genellina]

En Fri, 29 Jan 2010 13:53:43 -0300, Mr.SpOOn   
escribió:



I need to get keyboard input in a python program. I need it to let the
user choose some options, for example:

1) option 1
2) option 2
3) option 3

and then to input some data to the program.

I'm using the raw_input method and it works fine, but I noted that I
can't use backspace and when I accidentally press shift space (because
I need to input uppercase letters separated by a space) it writes some
strange characters.


That's strange. If you're using Linux, make sure you have the readline  
package installed.



So, is there another way to get keyboard input?


If readline capabilities are not enough, you could try Urwid, a library  
for writing console programs: http://excess.org/urwid/ - but it's a big  
move. wxPython would let you write a graphical interface (and it's a  
bigger move!)


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Thread safe locale techniques?

[python]

We're currently writing a web application based on a threaded
python web server framework (cherrypy) and would like to
simultaneously support users from multiple locales.

The locale module doesn't appear to be thread safe. Are there 3rd
party libraries or modules that provide locale parsing and
formatting functionality in a thread-safe way?

Suggestions appreciated!

Regards,
Malcolm
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Some C-API functions clear the error indicator?

[Austin Bingham]

Maybe I'm not following what you're saying. In my case, I already know
that an exception has been thrown. In the course of processing that
exception, I call another function which, for whatever reason and even
when it succeeds, clears the exception indicators. How can I address
this issue by checking function calls for failure?

Austin

On Fri, Jan 29, 2010 at 9:04 PM, Gabriel Genellina
 wrote:
> En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham
>  escribió:
>
>> I've noticed that several (many?) python functions seem to clear the
>> error/exception indicators when they're called from a C/C++ program.
>> For example, both PyImport_ImportModule and traceback.extract_tb()
>> (called via the function call methods) do this: if error indicators
>> are set prior to their call (as indicated by PyErr_Fetch, and
>> including a call to PyErr_Restore), I see that they are unset (using
>> the same method) after the call. This happens even when the functions
>> succeed.
>
> It's simple: you have to check *every* function call for failure. Many
> functions return new object references and you have to properly decrement
> them in case of failure, so in most cases this means that you have to check
> each and every call.
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Killing a Thread

[Gabriel Genellina]

En Thu, 28 Jan 2010 10:25:30 -0300, Richard Lamboj  
 escribió:


which Method is better to kill a Thread? Using Thread Events, or a  
raising a

Exception? Maybe someone has a small example for me?


The best way is simply NOT to do that. You don't kill a thread, you ask it  
to commit suicide.  There is no reliable way to forcefully shut down  
another thread.


The thread must periodically check some value (perhaps an Event object; in  
simple cases any variable will do) and (cleanly) exit when asked to.


Threads that run pure Python code may be interrupted using the API call  
PyThreadState_SetAsyncExc; look for a recipe using ctypes to call it from  
Python.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Changing tab behavior in Python Interactive mode

[Steve Holden]

Andrej Mitrovic wrote:
> On Jan 29, 6:47 pm, Steve Holden  wrote:
>> Andrej Mitrovic wrote:
>>> I've noticed that when running Python in interactive mode (via cmd on
>>> windows), the first time I hit tab it will move 4 spaces to the right,
>>> however each new tab will move 8 spaces instead of 4. Why this
>>> inconsistent behavior? And how could I change this to be consistent
>>> and always move only 4 spaces?
>> The first tab you are starting four characters in due to the interpreter
>> already having printed ">>> ".
>>
>> regards
>>  Steve
>> --
>> Steve Holden   +1 571 484 6266   +1 800 494 3119
>> PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
>> Holden Web LLChttp://www.holdenweb.com/
>> UPCOMING EVENTS:http://holdenweb.eventbrite.com/
> 
> I see. I wonder if I could modify this behavior.. I guess I'll have to
> take a look at the source. Thanks anyway!

I's nothing to do with the source: it isn't the interpreter that's
deciding where the "tab stops" are, it's the console window the
interpreter is running in.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python for net manager

[Gabriel Genellina]

En Wed, 27 Jan 2010 00:11:10 -0300, moon sky  escribió:


i just want to write a small tool to manage the network, including detect
the ip addr,netmask, and send the arp request to  find out locale's  
ip-mac

turtple,
is there any way to succeed this exlude 'subprocess call'?


Try scapy: http://www.secdev.org/projects/scapy/

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Changing tab behavior in Python Interactive mode

[Andrej Mitrovic]

On Jan 29, 6:47 pm, Steve Holden  wrote:
> Andrej Mitrovic wrote:
> > I've noticed that when running Python in interactive mode (via cmd on
> > windows), the first time I hit tab it will move 4 spaces to the right,
> > however each new tab will move 8 spaces instead of 4. Why this
> > inconsistent behavior? And how could I change this to be consistent
> > and always move only 4 spaces?
>
> The first tab you are starting four characters in due to the interpreter
> already having printed ">>> ".
>
> regards
>  Steve
> --
> Steve Holden           +1 571 484 6266   +1 800 494 3119
> PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
> Holden Web LLC                http://www.holdenweb.com/
> UPCOMING EVENTS:        http://holdenweb.eventbrite.com/

I see. I wonder if I could modify this behavior.. I guess I'll have to
take a look at the source. Thanks anyway!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[Gabriel Genellina]

En Fri, 29 Jan 2010 13:09:40 -0300, Michele Simionato  
 escribió:



On Jan 29, 2:30 pm, andrew cooke  wrote:

Is there any way to change the name of the function in an error
message?  In the example below I'd like the error to refer to bar(),
for example (the motivation is related function decorators - I'd like
the wrapper function to give the same name)


Use the decorator module which does the right thing:
http://pypi.python.org/pypi/decorator


The decorator module is a very fine addition to anyone's tool set -- but  
in this case it is enough to use the wraps() function from the functools  
standard module.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleXMLRPCServer daemon

[Gabriel Genellina]

En Fri, 29 Jan 2010 12:54:23 -0300, Thomas Allen   
escribió:



I have a script that runs an instance of SimpleXMLRPCServer and in
general it works as expected. In its __del__, it is supposed to clean
up its PID file (written on boot). I have two problems with this
server instance: The first is that tt doesn't always clean up its PID
file; is there a more reliable way to do this than how I am currently?
The second is that when it does crash, I don't know about it...what
would be sufficient as a "keep-alive" script to restart it? I suppose
I could use something like EventMachine (already installed on my
server) to watch the PID file if it were deleted reliably.


I agree with the recommendation of using some daemon library; doing it  
right is better left to the experts :)

But if you can't or don't want to alter your code so much, I suggest:

 - use atexit.register, instead of __del__, to delete the PID file
 - keep a lock on the pid file; if a second instance is able to write to  
it, it means it's an orphan from a previous, aborted run. (Checking  
whether a process with such pid exists is not enough - pids are recycled  
rather fast)


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: site.py confusion

[Gabriel Genellina]

En Wed, 27 Jan 2010 15:48:23 -0300, George Trojan   
escribió:

Arnaud Delobelle wrote:

George Trojan  writes:


Inspired by the 'Default path for files' thread I tried to use
sitecustomize in my code. What puzzles me is that the site.py's main()
is not executed. My sitecustomize.py is


That gave me the explanation why the above happens: when site is  
imported, the current directory is not yet prepended to sys.path.


I wanted to have library location specific to application without having  
to explicitly change sys.path in all App-Top-Dir/bin/*.py. I thought  
creating bin/sitecustomize.py would do the trick.


Put sitecustomize.py in any of those directories already in sys.path; on  
Python 2.6, you may put sitecustomize.py on your "user site directory",  
~/.local/lib/python2.6/site-packages (see PEP370; it won't show up in  
sys.path unless the directory actually exists)


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleXMLRPCServer daemon

[Sean DiZazzo]

On Jan 29, 7:54 am, Thomas Allen  wrote:
> I have a script that runs an instance of SimpleXMLRPCServer and in
> general it works as expected. In its __del__, it is supposed to clean
> up its PID file (written on boot). I have two problems with this
> server instance: The first is that tt doesn't always clean up its PID
> file; is there a more reliable way to do this than how I am currently?
> The second is that when it does crash, I don't know about it...what
> would be sufficient as a "keep-alive" script to restart it? I suppose
> I could use something like EventMachine (already installed on my
> server) to watch the PID file if it were deleted reliably.
>
> Thomas Allen

You should check out python-daemon.  I use home grown daemons all the
time, and have only played a little with the python-daemon library,
but it is surely written well, and handles lockfiles and pidfiles.

I believe you can map a function to any signal that the daemon
receives to do any kind of cleanup etc.  Ssometimes those PID files
might be left around, but the runner included with the module (i
think) checks for stale pid files.

There's all kinds of useful stuff.

In my home grown daemons I use Adam's technique, and wrap the "while
1:" block in a try except clause.

~Sean
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Some C-API functions clear the error indicator?

[Gabriel Genellina]

En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham  
 escribió:



I've noticed that several (many?) python functions seem to clear the
error/exception indicators when they're called from a C/C++ program.
For example, both PyImport_ImportModule and traceback.extract_tb()
(called via the function call methods) do this: if error indicators
are set prior to their call (as indicated by PyErr_Fetch, and
including a call to PyErr_Restore), I see that they are unset (using
the same method) after the call. This happens even when the functions
succeed.


It's simple: you have to check *every* function call for failure. Many  
functions return new object references and you have to properly decrement  
them in case of failure, so in most cases this means that you have to  
check each and every call.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Processing XML File

[Stefan Behnel]

Sells, Fred, 29.01.2010 20:31:
> Google is your friend.  Elementtree is one of the better documented
> IMHO, but there are many modules to do this.

Unless the OP provides some more information, "do this" is rather
underdefined. And sending someone off to Google who is just learning the
basics of Python and XML and trying to solve a very specific problem with
them is not exactly the spirit I'm used to in this newsgroup.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Processing XML File

[Sells, Fred]

Google is your friend.  Elementtree is one of the better documented
IMHO, but there are many modules to do this.

> -Original Message-
> From: python-list-bounces+frsells=adventistcare@python.org
> [mailto:python-list-bounces+frsells=adventistcare@python.org] On
> Behalf Of Stefan Behnel
> Sent: Friday, January 29, 2010 2:25 PM
> To: python-list@python.org
> Subject: Re: Processing XML File
> 
> jakecjacobson, 29.01.2010 18:25:
> > I need to take a XML web resource and split it up into smaller XML
> > files.  I am able to retrieve the web resource but I can't find any
> > good XML examples.  I am just learning Python so forgive me if this
> > question has been answered many times in the past.
> >
> > My resource is like:
> >
> > 
> >  ...
> >  ...
> > 
> > 
> >  ...
> >  ...
> > 
> 
> Is this what you get as a document or is this just /contained/ in the
> document?
> 
> Note that XML does not allow more than one root element, so the above
is
> not XML. Each of the two ... parts form an XML
> document by themselves, though.
> 
> 
> > So in this example, I would need to output 2 files with the contents
> > of each file what is between the open and close document tag.
> 
> Are the two files formatted as you show above? In that case, you can
> simply
> iterate over the lines and cut the document when you see "".
Or,
> if you are sure that "" only appears as top-most elements
and
> not
> inside of the documents, you can search for "" in the
content (a
> string, I guess) and split it there.
> 
> As was pointed out before, once you have these two documents, use the
> xml.etree package to work with them.
> 
> Something like this might work:
> 
> import xml.etree.ElementTree as ET
> 
> data = urllib2.urlopen(url).read()
> 
> for part in data.split(''):
> document = ET.fromstring(''+part)
> print(document.tag)
> # ... do other stuff
> 
> Stefan
> --
> http://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Processing XML File

[Stefan Behnel]

jakecjacobson, 29.01.2010 18:25:
> I need to take a XML web resource and split it up into smaller XML
> files.  I am able to retrieve the web resource but I can't find any
> good XML examples.  I am just learning Python so forgive me if this
> question has been answered many times in the past.
> 
> My resource is like:
> 
> 
>  ...
>  ...
> 
> 
>  ...
>  ...
> 

Is this what you get as a document or is this just /contained/ in the document?

Note that XML does not allow more than one root element, so the above is
not XML. Each of the two ... parts form an XML
document by themselves, though.


> So in this example, I would need to output 2 files with the contents
> of each file what is between the open and close document tag.

Are the two files formatted as you show above? In that case, you can simply
iterate over the lines and cut the document when you see "". Or,
if you are sure that "" only appears as top-most elements and not
inside of the documents, you can search for "" in the content (a
string, I guess) and split it there.

As was pointed out before, once you have these two documents, use the
xml.etree package to work with them.

Something like this might work:

import xml.etree.ElementTree as ET

data = urllib2.urlopen(url).read()

for part in data.split(''):
document = ET.fromstring(''+part)
print(document.tag)
# ... do other stuff

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Processing XML File

[Adam Tauno Williams]

On Fri, 2010-01-29 at 10:34 -0800, jakecjacobson wrote:
> On Jan 29, 1:04 pm, Adam Tauno Williams 
> wrote:
> > On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote:
> > > I need to take a XML web resource and split it up into smaller XML
> > > files.  I am able to retrieve the web resource but I can't find any
> > > good XML examples.  I am just learning Python so forgive me if this
> > > question has been answered many times in the past.
> > > My resource is like:
> > > 
> > >  ...
> > >  ...
> > > 
> > > 
> > > 
> > > So in this example, I would need to output 2 files with the contents
> > > of each file what is between the open and close document tag.
> > Do you want to parse the document or SaX?
> > I have a SaX example at
> > 
> Thanks but I am way over my head with XML, Python.  I am working with
> DDMS and need to output the individual resource nodes to their own
> file.  I hope that this helps and I need a good example and how to use
> it.


If that is all you need XPath will spit it apart for you like



doc = etree.parse(self._rfile)
results = doc.xpath(xpath)
for result in results:
  print str(result)

For example if your XML has an outermost element of ResultSet with inner row 
elements just do:
for record in doc.xpath(u'/ResultSet/row')

Implied import for these examples is "from lxml import etree"


> Here is what a resource node looks like:
>xsi:schemaLocation="https://metadata.dod.mil/mdr/ns/DDMS/1.4/
> https://metadata.dod.mil/mdr/ns/DDMS/1.4/";
> xmlns:ddms="https://metadata.dod.mil/mdr/ns/DDMS/1.4/";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:ICISM="urn:us:gov:ic:ism:v2">
> 
> https://metadata.dod.mil/mdr/
> ns/MDR/1.0/MDR.owl#GovernanceNamespace" ddms:value="TBD"/>
> 
>  ICISM:classification="U">Sample Taxonomy
>  ICISM:classification="U">
>   This is a sample taxonomy created for the Help page.
> 
> 
>  ICISM:classification="U">
>   
> Sample
> Developer
> FGM, Inc.
> 703-885-1000
> sampledevelo...@fgm.com
>   
> 
>  ICISM:classification="U" ICISM:nonICmarkings="DIST_STMT_A" />
> 
>   
> 
> You can see the DDMS site at https://metadata.dod.mil/.


-- 
OpenGroupware developer: awill...@whitemice.org

OpenGroupare & Cyrus IMAPd documenation @


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Processing XML File

[jakecjacobson]

On Jan 29, 1:04 pm, Adam Tauno Williams 
wrote:
> On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote:
> > I need to take a XML web resource and split it up into smaller XML
> > files.  I am able to retrieve the web resource but I can't find any
> > good XML examples.  I am just learning Python so forgive me if this
> > question has been answered many times in the past.
> > My resource is like:
> > 
> >      ...
> >      ...
> > 
> > 
> >      ...
> >      ...
> > 
> > So in this example, I would need to output 2 files with the contents
> > of each file what is between the open and close document tag.
>
> Do you want to parse the document or SaX?
>
> I have a SaX example at
> 

Thanks but I am way over my head with XML, Python.  I am working with
DDMS and need to output the individual resource nodes to their own
file.  I hope that this helps and I need a good example and how to use
it.

Here is what a resource node looks like:
  https://metadata.dod.mil/mdr/ns/DDMS/1.4/
https://metadata.dod.mil/mdr/ns/DDMS/1.4/";
xmlns:ddms="https://metadata.dod.mil/mdr/ns/DDMS/1.4/";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns:ICISM="urn:us:gov:ic:ism:v2">

https://metadata.dod.mil/mdr/
ns/MDR/1.0/MDR.owl#GovernanceNamespace" ddms:value="TBD"/>

Sample Taxonomy

  This is a sample taxonomy created for the Help page.



  
Sample
Developer
FGM, Inc.
703-885-1000
sampledevelo...@fgm.com
  



  

You can see the DDMS site at https://metadata.dod.mil/.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sleep timer but still responsive?

[JohnnyFive]

On Jan 29, 9:33 am, Andreas Tawn  wrote:
> > On Jan 28, 4:55 pm, "Gabriel Genellina" 
> > wrote:
> > > Please provide more details. What do you want your program to do
> > while
> > > sleeping? What kind of actions do you want a response to?
> > > Do you have a GUI? A curses-based interfase?
>
> > > --
> > > Gabriel Genellina
>
> > My app is purely console based. I just don't want the console to lock
> > up (on Windows using time.sleep(x) causes the console to become
> > unresponsive until the timer is done), and I want people to be able to
> > CTRL+C to stop the script if need be (which can't be done if it's
> > unresponsive!).
>
> > Thanks.
>
> How about this? Responds to ctrl+c, but still sleeps.
>
> import time
>
> def responsiveSleep(n):
>     while n > 0:
>         time.sleep(1)
>         n -= 1
>
> Cheers,
>
> Drea

Thanks for the ideas! Maybe it's just my computer, but using your
solution still causes the prompt to become unresponsive during the
sleeps.

I am using 2.6.4 btw. It's not a major deal though, I just thought
there had to be a way to do this fairly easily.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sleep timer but still responsive?

[Gabriel Genellina]

En Fri, 29 Jan 2010 14:39:31 -0300, MRAB   
escribió:

JohnnyFive wrote:



 My app is purely console based. I just don't want the console to lock
up (on Windows using time.sleep(x) causes the console to become
unresponsive until the timer is done), and I want people to be able to
CTRL+C to stop the script if need be (which can't be done if it's
unresponsive!).
 Thanks.


Which version of Python are you using? time.sleep(x) can be interrupted
in Python 2.6.


I'm able to be more precise: time.sleep() can be interrupted in any Python  
version since 2.3.


To the OP: beware of any unqualified 'except' clauses; a block like this:

  try:
...
  except:
do_something_or_pass

may "swallow" the KeyboardInterrupt exception (generated by a Ctrl-C  
press).
From Python 2.5 and up, the most generic exception clause should read  
`except Exception: ...`
In previous versions, you had to explicitely re-raise KeyboardInterrupt  
and SystemExit in any catch-all block:


  try:
...
  except (KeyboardInterrupt, SystemExit):
raise
  except:
...

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Processing XML File

[Adam Tauno Williams]

On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote:
> I need to take a XML web resource and split it up into smaller XML
> files.  I am able to retrieve the web resource but I can't find any
> good XML examples.  I am just learning Python so forgive me if this
> question has been answered many times in the past.
> My resource is like:
> 
>  ...
>  ...
> 
> 
>  ...
>  ...
> 
> So in this example, I would need to output 2 files with the contents
> of each file what is between the open and close document tag.

Do you want to parse the document or SaX?

I have a SaX example at


-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Sleep timer but still responsive?

[Andreas Tawn]

> On Jan 28, 4:55 pm, "Gabriel Genellina" 
> wrote:
> > Please provide more details. What do you want your program to do
> while
> > sleeping? What kind of actions do you want a response to?
> > Do you have a GUI? A curses-based interfase?
> >
> > --
> > Gabriel Genellina
> 
> My app is purely console based. I just don't want the console to lock
> up (on Windows using time.sleep(x) causes the console to become
> unresponsive until the timer is done), and I want people to be able to
> CTRL+C to stop the script if need be (which can't be done if it's
> unresponsive!).
> 
> Thanks.

How about this? Responds to ctrl+c, but still sleeps.

import time

def responsiveSleep(n):
while n > 0:
time.sleep(1)
n -= 1

Cheers,

Drea
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Changing tab behavior in Python Interactive mode

[Steve Holden]

Andrej Mitrovic wrote:
> I've noticed that when running Python in interactive mode (via cmd on
> windows), the first time I hit tab it will move 4 spaces to the right,
> however each new tab will move 8 spaces instead of 4. Why this
> inconsistent behavior? And how could I change this to be consistent
> and always move only 4 spaces?

The first tab you are starting four characters in due to the interpreter
already having printed ">>> ".

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sleep timer but still responsive?

[MRAB]

JohnnyFive wrote:

On Jan 28, 4:55 pm, "Gabriel Genellina" 
wrote:
Please provide more details. What do you want your program to do while  
sleeping? What kind of actions do you want a response to?

Do you have a GUI? A curses-based interfase?

--
Gabriel Genellina


My app is purely console based. I just don't want the console to lock
up (on Windows using time.sleep(x) causes the console to become
unresponsive until the timer is done), and I want people to be able to
CTRL+C to stop the script if need be (which can't be done if it's
unresponsive!).

Thanks.


Which version of Python are you using? time.sleep(x) can be interrupted
in Python 2.6.

If the version you're using can't be interrupted then you could use
multiple sleeps:

# Wait for a total of 10 secs.
for i in range(10):
time.sleep(1)
--
http://mail.python.org/mailman/listinfo/python-list

Changing tab behavior in Python Interactive mode

[Andrej Mitrovic]

I've noticed that when running Python in interactive mode (via cmd on
windows), the first time I hit tab it will move 4 spaces to the right,
however each new tab will move 8 spaces instead of 4. Why this
inconsistent behavior? And how could I change this to be consistent
and always move only 4 spaces?
-- 
http://mail.python.org/mailman/listinfo/python-list

Processing XML File

[jakecjacobson]

I need to take a XML web resource and split it up into smaller XML
files.  I am able to retrieve the web resource but I can't find any
good XML examples.  I am just learning Python so forgive me if this
question has been answered many times in the past.

My resource is like:


 ...
 ...


 ...
 ...


So in this example, I would need to output 2 files with the contents
of each file what is between the open and close document tag.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: get PyObject* knowing its string name

[Robert Kern]

On 2010-01-29 11:00 AM, Mr.M wrote:

I think this sounds like a stupid question, but I searched the C/Api doc
and google but I wasn't able to find any hint:

how can I retrive PyObject pointer if I only know it's name?

What I'd like to do is something like this:

[code]
PyObject* obj = PyFindWhatImLookingFor("w.z.y.x");
[/code]

Of course if I know that w is a module, z is a class and so on I can
search w dictionary than z members than...

But I wonder if there is a way to retrive without knowing what w.z.y.x
means.


There is probably a clever/careful way to do it, but most systems that do this 
just separate the package/module parts from the object parts:


  "w:z.y.x"

Import the module part, then getattr() your way down the object part.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Sleep timer but still responsive?

[JohnnyFive]

On Jan 28, 4:55 pm, "Gabriel Genellina" 
wrote:
> Please provide more details. What do you want your program to do while  
> sleeping? What kind of actions do you want a response to?
> Do you have a GUI? A curses-based interfase?
>
> --
> Gabriel Genellina

My app is purely console based. I just don't want the console to lock
up (on Windows using time.sleep(x) causes the console to become
unresponsive until the timer is done), and I want people to be able to
CTRL+C to stop the script if need be (which can't be done if it's
unresponsive!).

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

get PyObject* knowing its string name

[Mr.M]

I think this sounds like a stupid question, but I searched the C/Api doc 
and google but I wasn't able to find any hint:


how can I retrive PyObject pointer if I only know it's name?

What I'd like to do is something like this:

[code]
PyObject* obj = PyFindWhatImLookingFor("w.z.y.x");
[/code]

Of course if I know that w is a module, z is a class and so on I can 
search w dictionary than z members than...


But I wonder if there is a way to retrive without knowing what w.z.y.x 
means.


Thank you,

Luca.
--
http://mail.python.org/mailman/listinfo/python-list

Keyboard input

[Mr.SpOOn]

Hi,
I need to get keyboard input in a python program. I need it to let the
user choose some options, for example:

1) option 1
2) option 2
3) option 3

and then to input some data to the program.

I'm using the raw_input method and it works fine, but I noted that I
can't use backspace and when I accidentally press shift space (because
I need to input uppercase letters separated by a space) it writes some
strange characters.

So, is there another way to get keyboard input?

Thanks,
Carlo
-- 
http://mail.python.org/mailman/listinfo/python-list

Threading issue with SQLite

[Alan Harris-Reid]

Hi,

I am creating a web application (using Python 3.1 and CherryPy 3.2) 
where a SQLite connection and cursor object are created using the 
following code (simplified from the original):


class MainSite:
   con = sqlite.connect('MyDatabase.db')
   cursor = con.cursor()

   def index_page():
  some HTML code
  cursor.execute(some SQL statement)
  more HTML code
 
   def another_page():

  some HTML code
  cursor.execute(anotherSQL statement)
  more HTML code

When I call a URL which launches the another_page() method I get the 
error message "sqlite3.ProgrammingError: SQLite objects created in a 
thread can only be used in that same thread."


Questions...
1.  Is there a large overhead in opening a new SQLite connection for 
each thread (ie. within each method)?
2.  Is there any way to use the same connection for the whole class (or 
should I forget that idea completely?)
3.  When a method returns to the calling method, is the connection 
automatically closed (assuming the object is local, of course) or does 
it have to be done explicitly using connection.close()?


TIA,
Alan Harris-Reid

--
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleXMLRPCServer daemon

[Tim Wintle]

On Fri, 2010-01-29 at 07:54 -0800, Thomas Allen wrote:
> The second is that when it does crash, I don't know about it...what
> would be sufficient as a "keep-alive" script to restart it? I suppose
> I could use something like EventMachine (already installed on my
> server) to watch the PID file if it were deleted reliably.

If the server crashes then it clearly won't get around to deleting it's
pid file.

The way I do it is to use os.kill (with signal 0 IIRC) to check if the
process is still alive when the script starts. If it's not then I delete
the pid file and carry on starting up.

Tim

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Funny behaviour with __future__ and doctest between 2.6 and 3.1

[Peter Otten]

Mattsteel wrote:

> Sadly (for me), you're right... then the only way to use doctest to
> work both in 2.6 and 3.1 (without modifications between them) is
> something like this:
> 
> #!/usr/bin/env python
> '''
 str(concat('hello','world'))
> 'hello world'
> '''
> from __future__  import unicode_literals
> def concat( first, second ):
> return first + ' ' + second
> if __name__ == "__main__":
> import doctest
> doctest.testmod()
> 
> Is there any way to avoid using str(...) to protect the string?
> M.

I think you can work around the problem. The following should pass in Python 
2.6 and 3.1:

'''
>>> concat('hello','world') == 'hello world'
True
'''
from __future__  import unicode_literals

def concat( first, second ):
return first + ' ' + second

if __name__ == "__main__":
import doctest
doctest.testmod()

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Funny behaviour with __future__ and doctest between 2.6 and 3.1

[Mattsteel]

Hi Peter.
Sadly (for me), you're right... then the only way to use doctest to
work both in 2.6 and 3.1 (without modifications between them) is
something like this:

#!/usr/bin/env python
'''
>>> str(concat('hello','world'))
'hello world'
'''
from __future__  import unicode_literals
def concat( first, second ):
return first + ' ' + second
if __name__ == "__main__":
import doctest
doctest.testmod()

Is there any way to avoid using str(...) to protect the string?
M.


---
 #!/usr/bin/env python
'''
>>> concat('hello','world')
'hello world'
'''
from __future__  import unicode_literals
def concat( first, second ):
return first + ' ' + second
if __name__ == "__main__":
import doctest
doctest.testmod()


On 29 Gen, 16:50, Peter Otten <__pete...@web.de> wrote:
> Mattsteel wrote:
> > Hello all.
> > I'm using Python 2.6.4 and Python 3.1.1.
> > My wish is to code in a 3.1-compliant way using 2.6, so I'm importing
> > the __future__ module.
> > I've found a funny thing comparing the two folliwing snippets that
> > differ for one line only, that is the position of __future__ import
> > (before or after the doc string).
>
> > Well, I understand the subtle difference but still I wander what
> > really happen behind the scenes.
>
> Are you sure? The second script has no module docstring, just a string
> literal somewhere in the module, and therefore no tests. You can see that by
> running it with the verbose option -v. Also,
>
> from __future__ import unicode_literals
>
> doesn't affect the repr() of a unicode instance. But the interactive
> interpreter invokes repr() on the result before it is printed:
>
> >>> from __future__ import unicode_literals
> >>> "yadda"
>
> u'yadda'
>
> > Comments are welcome.
>
> > ---
> >     #!/usr/bin/env python
> >     '''
> >     >>> concat('hello','world')
> >     'hello world'
> >     '''
> >     from __future__  import unicode_literals
> >     def concat( first, second ):
> >         return first + ' ' + second
> >     if __name__ == "__main__":
> >         import doctest
> >         doctest.testmod()
> > ---
> >     #!/usr/bin/env python
> >     from __future__  import unicode_literals
> >     '''
> >     >>> concat('hello','world')
> >     'hello world'
> >     '''
> >     def concat( first, second ):
> >         return first + ' ' + second
> >     if __name__ == "__main__":
> >         import doctest
> >         doctest.testmod()
> > ---
>
> > The first way shows the following failure:
>
> > ---
> >   Failed example:
> >       concat('hello','world')
> >   Expected:
> >       'hello world'
> >   Got:
> >       u'hello world'
>
> > ---
>
> > Regards.
>
> > Matt.
>
>

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[Michele Simionato]

On Jan 29, 2:30 pm, andrew cooke  wrote:
> Is there any way to change the name of the function in an error
> message?  In the example below I'd like the error to refer to bar(),
> for example (the motivation is related function decorators - I'd like
> the wrapper function to give the same name)

Use the decorator module which does the right thing:
http://pypi.python.org/pypi/decorator
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleXMLRPCServer daemon

[Adam Tauno Williams]

On Fri, 2010-01-29 at 07:54 -0800, Thomas Allen wrote:
> I have a script that runs an instance of SimpleXMLRPCServer and in
> general it works as expected. In its __del__, it is supposed to clean
> up its PID file (written on boot). I have two problems with this
> server instance: The first is that tt doesn't always clean up its PID
> file; is there a more reliable way to do this than how I am currently?
> The second is that when it does crash, I don't know about it...what
> would be sufficient as a "keep-alive" script to restart it? I suppose
> I could use something like EventMachine (already installed on my
> server) to watch the PID file if it were deleted reliably.

Why don't you wrap the server in a try/except block?  Then if an
exception occurs killing the server you can automatically restart and/or
send an e-mail to an admin address.

-- 
http://mail.python.org/mailman/listinfo/python-list

SimpleXMLRPCServer daemon

[Thomas Allen]

I have a script that runs an instance of SimpleXMLRPCServer and in
general it works as expected. In its __del__, it is supposed to clean
up its PID file (written on boot). I have two problems with this
server instance: The first is that tt doesn't always clean up its PID
file; is there a more reliable way to do this than how I am currently?
The second is that when it does crash, I don't know about it...what
would be sufficient as a "keep-alive" script to restart it? I suppose
I could use something like EventMachine (already installed on my
server) to watch the PID file if it were deleted reliably.

Thomas Allen
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Funny behaviour with __future__ and doctest between 2.6 and 3.1

[Peter Otten]

Mattsteel wrote:

> Hello all.
> I'm using Python 2.6.4 and Python 3.1.1.
> My wish is to code in a 3.1-compliant way using 2.6, so I'm importing
> the __future__ module.
> I've found a funny thing comparing the two folliwing snippets that
> differ for one line only, that is the position of __future__ import
> (before or after the doc string).
> 
> Well, I understand the subtle difference but still I wander what
> really happen behind the scenes.

Are you sure? The second script has no module docstring, just a string 
literal somewhere in the module, and therefore no tests. You can see that by 
running it with the verbose option -v. Also,

from __future__ import unicode_literals 

doesn't affect the repr() of a unicode instance. But the interactive 
interpreter invokes repr() on the result before it is printed:

>>> from __future__ import unicode_literals
>>> "yadda"
u'yadda'

> Comments are welcome.
> 
> ---
> #!/usr/bin/env python
> '''
> >>> concat('hello','world')
> 'hello world'
> '''
> from __future__  import unicode_literals
> def concat( first, second ):
> return first + ' ' + second
> if __name__ == "__main__":
> import doctest
> doctest.testmod()
> ---
> #!/usr/bin/env python
> from __future__  import unicode_literals
> '''
> >>> concat('hello','world')
> 'hello world'
> '''
> def concat( first, second ):
> return first + ' ' + second
> if __name__ == "__main__":
> import doctest
> doctest.testmod()
> ---
> 
> 
> The first way shows the following failure:
> 
> ---
>   Failed example:
>   concat('hello','world')
>   Expected:
>   'hello world'
>   Got:
>   u'hello world'
> 
> ---
> 
> Regards.
> 
> Matt.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[eric_dex...@msn.com]

On Jan 27, 2:56 pm, John Nagle  wrote:
> Daniel Fetchinson wrote:
> > Hi folks,
>
> > I was going to write this post for a while because all sorts of myths
> > periodically come up on this list about python 3. I don't think the
> > posters mean to spread false information on purpose, they simply are
> > not aware of the facts.
>
> > My list is surely incomplete, please feel free to post your favorite
> > misconception about python 3 that people periodically state, claim or
> > ask about.
>
> Myths about Python 3:
>
> 1.  Python 3 is supported by major Linux distributions.
>
>         FALSE - most distros are shipping with Python 2.4, or 2.5 at best.
>
> 2.  Python 3 is supported by multiple Python implementations.
>
>         FALSE - Only CPython supports 3.x.  Iron Python, Unladen Swallow,
>         PyPy, and Jython have all stayed with 2.x versions of Python.
>
> 3.  Python 3 is supported by most 3rd party Python packages.
>
>         FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc.
>
> Arguably, Python 3 has been rejected by the market.  Instead, there's
> now Python 2.6, Python 2.7, and Python 2.8.  Python 3 has turned into
> a debacle like Perl 6, now 10 years old.
>
> That's the reality, Python 3 fanboys.
>
>                                 John Nagle

I am seeing alot of bleeding edge linux distro's with 2.5 and 2.6 I
would perfer they stayed with 2.4 or 2.5 ..
or if there is a version that I can come up with a convincing cross-
platform os.startfile I would want to use
a vmware version of that linux.
___
http://dextracker.blogspot.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 3's adoption

[Alf P. Steinbach]

* Steven D'Aprano:

On Fri, 29 Jan 2010 07:10:01 +0100, Alf P. Steinbach wrote:


   >>> L = ["æ", "ø", "å"]   # This is in SORTED ORDER in Norwegian L


[...]


   >>> L.sort( key = locale.strxfrm )
   >>> L
   ['å', 'æ', 'ø']
   >>> locale.strcoll( "å", "æ" )
   1
   >>> locale.strcoll( "æ", "ø" )
   -1

Note that strcoll correctly orders the strings as ["æ", "ø", "å"], that
is, it would have if it could have been used as cmp function to sort (or
better, to a separate routine named e.g. custom_sort).


This is in Python2.5, so I'm explicitly specifying unicode strings:


L = [u"æ", u"ø", u"å"]
assert sorted(L) == [u'å', u'æ', u'ø']


The default C-locale sorting of L does not equal to L. Now let's change 
to Norwegian locale:



import locale
locale.setlocale(locale.LC_ALL, 'nb_NO')

'nb_NO'

print u''.join(sorted(L, cmp=locale.strcoll))

æøå

So far so good, we've confirmed that in Python 2.5 we can sort in a 
locale-aware form. Now, can we do this with a key function? Thanks to 
Raymond Hettinger's recipe here:


http://code.activestate.com/recipes/576653/



print u''.join(sorted(L, key=CmpToKey(locale.strcoll)))

æøå


Success!

Let's try it in Python 3.1:


L = ["æ", "ø", "å"]
assert sorted(L) == ['å', 'æ', 'ø']

import locale
locale.setlocale(locale.LC_ALL, 'nb_NO')

'nb_NO'



Hm. A bit off-topic, but...

  >>> import locale
  >>> locale.setlocale(locale.LC_ALL, 'nb_NO')
  Traceback (most recent call last):
File "", line 1, in 
File "C:\Program Files\cpython\python31\lib\locale.py", line 527, in
  return _setlocale(category, locale)
  locale.Error: unsupported locale setting
  >>> _

This on a machine where the Python default locale is Norwegian.



''.join(sorted(L, key=CmpToKey(locale.strcoll)))

'æøå'


The definition of CmpToKey can be found at the URL above. It's not very 
exciting, but here it is:



def CmpToKey(mycmp):
'Convert a cmp= function into a key= function'
class K(object):
def __init__(self, obj, *args):
self.obj = obj
def __lt__(self, other):
return mycmp(self.obj, other.obj) == -1
def __gt__(self, other):
return mycmp(self.obj, other.obj) == 1
def __eq__(self, other):
return mycmp(self.obj, other.obj) == 0
def __le__(self, other):
return mycmp(self.obj, other.obj) != 1  
def __ge__(self, other):

return mycmp(self.obj, other.obj) != -1
def __ne__(self, other):
return mycmp(self.obj, other.obj) != 0
return K


This is pretty smart as a generic solution.

Thanks! :-)

I was thinking more of sticking those comparisions in some custom string class 
or such, which would be rather ugly...


The above does have quite high overhead though!

That is, it's /inefficient/ compared to using a 'cmp' function directly.


If that's too verbose for you, stick this as a helper function in your 
application:



def CmpToKey(mycmp):
'Convert a cmp= function into a key= function'
class K(object):
def __init__(self, obj, *args):
self.obj = obj
__lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1
__gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1
__eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0
__le__ = lambda s, o: mycmp(s.obj, o.obj) != 1
__ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1
__ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0
return K


[...]

The above may just be a bug in the 3.x stxfrm. But it illustrates that
sometimes you have your sort order defined by a comparision function.
Transforming that into a key can be practically impossible (it can also
be quite inefficient).


This might be true, but you haven't demonstrated it.


The "can be ... practically impossible" was hogwash. I posted late, sorry. The 
"quite inefficient" holds.



With one little 
helper function, you should be able to convert any comparison function 
into a key function, with relatively little overhead.


I wouldn't call an extra Python method call per comparision "relatively little 
overhead".


Note that the same number of comparisions as with a 'cmp' based sort is 
performed.

But with the wrapper every comparision is indirected through a Python method 
call (I think, but not sure, that the creation of those comparision objects does 
not introduce significant overhead, but the calls must).




# -*- coding: utf-8 -*-
from __future__ import print_function
from __future__ import unicode_literals
try:
range = xrange
except:
pass


import locale
import sys
import random
import timeit

def CmpToKey( mycmp ):
"Convert a cmp= function into a key= function"
class K( object ):
def __init__( self, obj, *args ):
self.obj = obj
def __lt__( self, other ):
return mycmp( self.obj, other.obj ) == -1
def __gt__( self, other ):
return mycmp( self.obj, other.obj ) == 1
def __eq__( self, other ):
return mycmp( 

Re: python 3's adoption

[Paul Boddie]

On 29 Jan, 06:56, Terry Reedy  wrote:
> On 1/28/2010 6:47 PM, Paul Boddie wrote:
>
> > What would annoy me if I used Python 3.x would be the apparent lack of
> > the __cmp__ method for conveniently defining comparisons between
> > instances of my own classes. Having to define all the rich comparison
> > methods frequently isn't even as much fun as it sounds.
>
> You define __eq__, which automatically generates __ne__.

>From the Python 3.1 language reference:

"There are no implied relationships among the comparison operators.
The truth of x==y does not imply that x!=y is false. Accordingly, when
defining __eq__(), one should also define __ne__() so that the
operators will behave as expected."

Maybe Python 3.1 plugs a default method in, anyway.

> You define __lt__, which is all sort and heap need.
> This should be about as easier than __eq__, which is really needed, and
> __cmp__. If you need the other 3, either copy the recipe in the
> Cookbook, or, even faster
>
> def __ge__(s,o): return o.__lt__(s)
> def __le__(s,o): return s < o or s == o
> def __ge__(s,o): return s > o or s == o

Spot the error in the above. ;-) Of course, this could be defined in a
base class and be inherited everywhere, but the case that made me
raise this issue actually involved instances of a class delegating
comparison/sorting to another object. With __cmp__ this can be done
concisely and only involve the delegation of one method - not so with
the rich comparison "protocol".

Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Funny behaviour with __future__ and doctest between 2.6 and 3.1

[Mattsteel]

Hello all.
I'm using Python 2.6.4 and Python 3.1.1.
My wish is to code in a 3.1-compliant way using 2.6, so I'm importing
the __future__ module.
I've found a funny thing comparing the two folliwing snippets that
differ for one line only, that is the position of __future__ import
(before or after the doc string).

Well, I understand the subtle difference but still I wander what
really happen behind the scenes.
Comments are welcome.

---
#!/usr/bin/env python
'''
>>> concat('hello','world')
'hello world'
'''
from __future__  import unicode_literals
def concat( first, second ):
return first + ' ' + second
if __name__ == "__main__":
import doctest
doctest.testmod()
---
#!/usr/bin/env python
from __future__  import unicode_literals
'''
>>> concat('hello','world')
'hello world'
'''
def concat( first, second ):
return first + ' ' + second
if __name__ == "__main__":
import doctest
doctest.testmod()
---


The first way shows the following failure:

---
  Failed example:
  concat('hello','world')
  Expected:
  'hello world'
  Got:
  u'hello world'

---

Regards.

Matt.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Some C-API functions clear the error indicator?

[Duncan Booth]

Austin Bingham  wrote:

> The functions that do this don't seem to indicate in their
> documentation that this will happen. So first, does anyone know why
> this is happening? Is it because of the context in which I'm making
> the calls? Is there any pattern or reason behind which functions will
> do this? Or am I just doing something wrong?
> 
(Just guessing here)
I would expect that any function that executes Python code will clear the 
error.

I think that has to happen otherwise the Python code will throw an 
exception whenever it gets round to checking for errors. In the past I've 
found that if you fail to check for an error in C code before returning to 
the interpreter you get the exception thrown a few instructions later, so 
something similar would probably happen if you call other Python code from 
C.

If it is anything that executes Python then that would include any function 
that creates or destroys an object with Python constructor or destructor 
code. or that compares or otherwise operates on instances defined in 
Python. In particular it might mean that any function that doesn't appear 
to clear the error could do so in a slightly different situation.

-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[exarkun]

On 02:10 pm, c...@rebertia.com wrote:
On Fri, Jan 29, 2010 at 5:30 AM, andrew cooke  
wrote:

Is there any way to change the name of the function in an error
message? �In the example below I'd like the error to refer to bar(),
for example (the motivation is related function decorators - I'd like
the wrapper function to give the same name)

def foo():

... � � return 7
...

foo.__name__ = 'bar'
foo(123)

Traceback (most recent call last):
�File "", line 1, in 
TypeError: foo() takes no arguments (1 given)


It gets weirder:

print(foo)




The name is represented in (at least) two places, on the function object 
and on the code object:


   >>> def foo(): pass
   ...>>> foo.func_name
   'foo'
   >>> foo.func_code.co_name
   'foo'
   >>> foo.func_name = 'bar'
   >>> foo
   
   >>> foo.func_code.co_name = 'baz'
   Traceback (most recent call last):
 File "", line 1, in 
   TypeError: readonly attribute
   >>>
new.function and new.code will let you construct new objects with 
different values (and copying over whichever existing attributes you 
want to preserve).


Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a program

[nn]

Johann Spies wrote:
> On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote:
> > Hi folks,
> >
> > I am a newbie to python, and I would be grateful if someone could
> > point out the mistake in my program. Basically, I have a huge text
> > file similar to the format below:
> >
> > AGACTCGAGTGCGCGGA   0
> > AGATAAGCTAATTAAGCTACTGG 0
> > AGATAAGCTAATTAAGCTACTGGGTT   1
> > AGCTCACAATAT 1
> > AGGTCGCCTGACGGCTGC  0
>
> I know this is a python list but if you really want to get the job
> done quickly this is one method without writing python code:
>
> $ cat /tmp/y
> AGACTCGAGTGCGCGGA   0
> AGATAAGCTAATTAAGCTACTGG 0
> AGATAAGCTAATTAAGCTACTGGGTT   1
> AGCTCACAATAT 1
> AGGTCGCCTGACGGCTGC  0
> $ grep -v 0 /tmp/y > tmp/z
> $ cat /tmp/z
> AGATAAGCTAATTAAGCTACTGGGTT   1
> AGCTCACAATAT 1
>
> Regards
> Johann
> --
> Johann Spies  Telefoon: 021-808 4599
> Informasietegnologie, Universiteit van Stellenbosch
>
>  "My son, if sinners entice thee, consent thou not."
> Proverbs 1:10

I would rather use awk for this:

awk 'NF==2 && $2!~/^0$/ {printf("seq%s\n%s\n",NR,$1)}' dnain.dat

but I think that is getting a bit off topic...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.1 simplejson install

[Chris Rebert]

On Fri, Jan 29, 2010 at 6:33 AM, Peter Otten <__pete...@web.de> wrote:
> dirknbr wrote:
>> I am trying to install simplejson on Python 3.1 on Windows. When I do
>> 'python setup.py install' I get 'except DisutilsPlatformError, x:
>> SyntaxError' with a dash under the comma.
>
> You are trying to install a package written for Python 2.x on 3.x, and some
> of the 2.x Syntax is illegal in Python 3.

To be specific (http://www.python.org/dev/peps/pep-3110/):

"the ambiguous
except A, B:
which would mean different things in Python 2.x and 3.x -- leading to
hard-to-catch bugs -- cannot legally occur in 3.x code."

The except statement in your error would be properly written in 3.x as:
except DisutilsPlatformError as x:

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.1 simplejson install

[Christian Heimes]

dirknbr wrote:
> I am trying to install simplejson on Python 3.1 on Windows. When I do
> 'python setup.py install' I get 'except DisutilsPlatformError, x:
> SyntaxError' with a dash under the comma.

There is no need to install simplejson yourself. Python 3 and 2.6
already have a JSON package called "json".

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Some C-API functions clear the error indicator?

[Austin Bingham]

I've noticed that several (many?) python functions seem to clear the
error/exception indicators when they're called from a C/C++ program.
For example, both PyImport_ImportModule and traceback.extract_tb()
(called via the function call methods) do this: if error indicators
are set prior to their call (as indicated by PyErr_Fetch, and
including a call to PyErr_Restore), I see that they are unset (using
the same method) after the call. This happens even when the functions
succeed.

The functions that do this don't seem to indicate in their
documentation that this will happen. So first, does anyone know why
this is happening? Is it because of the context in which I'm making
the calls? Is there any pattern or reason behind which functions will
do this? Or am I just doing something wrong?

If the problem is context-dependent (e.g. I haven't properly
established a call stack, or something of that flavor), any pointers
on doing things properly would be great.

Here's some example code demonstrating the problem:

---

  #include 

  int main(int argc, char** argv)
  {
Py_Initialize();

// Cause an IndexError
PyObject* list = PyList_New(0);
PyObject* obj = PyList_GetItem(list, 100);

PyObject *t = NULL, *v = NULL, *tb = NULL;

// Verify that we see the error
PyErr_Fetch(&t, &v, &tb);
assert(t);
PyErr_Restore(t, v, tb);

// Import a module, which seems to be clearing the error indicator
PyObject* mod = PyImport_ImportModule("sys");
assert(PyObject_HasAttrString(mod, "path"));

// Verify that the error indicator has been cleared
PyErr_Fetch(&t, &v, &tb);
assert(!t); // <=== The error is gone!
PyErr_Restore(t, v, tb);

Py_Finalize();

return 0;
  }

---

Thanks in advance.

Austin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.1 simplejson install

[Peter Otten]

dirknbr wrote:

> I am trying to install simplejson on Python 3.1 on Windows. When I do
> 'python setup.py install' I get 'except DisutilsPlatformError, x:
> SyntaxError' with a dash under the comma.

You are trying to install a package written for Python 2.x on 3.x, and some 
of the 2.x Syntax is illegal in Python 3.

I recommend that you use the json module instead that is already included in 
the 3.1 distribution, see

http://docs.python.org/3.1/library/json.html

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[Peter Otten]

andrew cooke wrote:

> Is there any way to change the name of the function in an error
> message?  In the example below I'd like the error to refer to bar(),
> for example (the motivation is related function decorators - I'd like
> the wrapper function to give the same name)
> 
 def foo():
> ... return 7
> ...
 foo.__name__ = 'bar'
 foo(123)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: foo() takes no arguments (1 given)

The name is looked up in the code object. As that is immutable you have to 
make a new one:

argnames = 'argcount nlocals stacksize flags code consts names varnames 
filename name firstlineno lnotab'.split()

def f(): return 42

code = type(f.func_code)
function = type(f)

def make_code(proto, **kw):
for name in argnames:
if name not in kw:
kw[name] = getattr(proto, "co_" + name)
values = [kw[name] for name in argnames]
return code(*values)

if __name__ == "__main__":
def foo():
print "foo"

c = make_code(foo.func_code, name="bar")
foo.func_code = c

foo(42)

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a program

[D'Arcy J.M. Cain]

On Fri, 29 Jan 2010 11:23:54 +0200
Johann Spies  wrote:
> I know this is a python list but if you really want to get the job
> done quickly this is one method without writing python code:
> [...]
> $ grep -v 0 /tmp/y > tmp/z

There's plenty of ways to do it without writing Python.  C, C++, Perl,
Forth, Awk, BASIC, Intercal, etc.  So what?  Besides, your solution
doesn't work.  You want "grep -vw 0 /tmp/y > tmp/z" and even then it
doesn't meet the requirements.  It extracts the lines the OP wants but
doesn't reformat them.  It also assumes a Unix system or at least
something with grep installed so it isn't portable.

If you want to see how the same task can be done in many different
languages see http://www.roesler-ac.de/wolfram/hello.htm.

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Python 3.1 simplejson install

[dirknbr]

I am trying to install simplejson on Python 3.1 on Windows. When I do
'python setup.py install' I get 'except DisutilsPlatformError, x:
SyntaxError' with a dash under the comma.

Any ideas?

Dirk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[Chris Rebert]

On Fri, Jan 29, 2010 at 5:30 AM, andrew cooke  wrote:
> Is there any way to change the name of the function in an error
> message?  In the example below I'd like the error to refer to bar(),
> for example (the motivation is related function decorators - I'd like
> the wrapper function to give the same name)
>
 def foo():
> ...     return 7
> ...
 foo.__name__ = 'bar'
 foo(123)
> Traceback (most recent call last):
>  File "", line 1, in 
> TypeError: foo() takes no arguments (1 given)

It gets weirder:

>>> print(foo)


Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 3's adoption

[Duncan Booth]

Steven D'Aprano  wrote:

> If that's too verbose for you, stick this as a helper function in your 
> application:
> 
> 
> def CmpToKey(mycmp):
> 'Convert a cmp= function into a key= function'
> class K(object):
> def __init__(self, obj, *args):
> self.obj = obj
> __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1
> __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1
> __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0
> __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1
> __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1
> __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0
> return K
> 
> 

Shouldn't that be:

def CmpToKey(mycmp):
'Convert a cmp= function into a key= function'
class K(object):
def __init__(self, obj, *args):
self.obj = obj
__lt__ = lambda s, o: mycmp(s.obj, o.obj) < 0
__gt__ = lambda s, o: mycmp(s.obj, o.obj) > 0
__eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0
__le__ = lambda s, o: mycmp(s.obj, o.obj) <= 0
__ge__ = lambda s, o: mycmp(s.obj, o.obj) >= 0
__ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0
return K

After all the cmp function needn't return -1,0,1 only negative, 0, 
positive.

And if that's still too verbose for you try:

def CmpToKey(mycmp):
'Convert a cmp= function into a key= function'
class K(object):
def __init__(self, obj, *args):
self.obj = obj
__lt__ = lambda s, o: mycmp(s.obj, o.obj) < 0
return K

That works on 3.1 and 2.6 though of course there's no guarantee that a 
future Python version won't use a different comparison operator so only 
use it if you like living dangerously and have problems typing. :^)


-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Benjamin Kaplan]

On Fri, Jan 29, 2010 at 8:13 AM, Anssi Saari  wrote:
> Daniel Fetchinson  writes:
>
>>> 1.  Python 3 is supported by major Linux distributions.
>>>
>>>      FALSE - most distros are shipping with Python 2.4, or 2.5 at best.
>>
>> This latter statement is false, Fedora 11 and 12 come with python 2.6.
>
> How does your mention of one distro counter that claim? Personally,
> I'd like to see a study of what version of Python ships with what
> Linux distribution. Say, include the top 100 distros from
> distrowatch.com? I think there may a surprising number of distros that
> ship with no version of Python what so ever.
> --

Just from a quick look, 6 of the top 10 distros on distrowatch have
Python 2.6 including all of the top 4.

> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Duncan Booth]

Anssi Saari  wrote:

> Daniel Fetchinson  writes:
> 
>>> 1.  Python 3 is supported by major Linux distributions.
>>>
>>>  FALSE - most distros are shipping with Python 2.4, or 2.5 at best.
>>
>> This latter statement is false, Fedora 11 and 12 come with python 2.6.
> 
> How does your mention of one distro counter that claim? Personally,
> I'd like to see a study of what version of Python ships with what
> Linux distribution. Say, include the top 100 distros from
> distrowatch.com? I think there may a surprising number of distros that
> ship with no version of Python what so ever.
> 
Is there any easier way to build that list than working through all the 
distributions by hand?

Maybe someone should create a wiki page to record this.

Here's what I see in the Ubuntu packages. Python 3 seems only to be in the 
universe repositories so far.

Dapper: Python 2.4.2
Hardy: Python 2.5.2
Intrepid: Python 2.5.2, 3.0~b3 (universe)
Jaunty: Python 2.6.2, 3.0.1 (universe)
Karmic: Python 2.6.4rc1, 3.1 (universe)
Lucid: Python 2.6.4, 3.1 (universe)

WTF is rc anything doing in the main repository?

-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function name unchanged in error message

[Jean-Michel Pichavant]

andrew cooke wrote:

Is there any way to change the name of the function in an error
message?  In the example below I'd like the error to refer to bar(),
for example (the motivation is related function decorators - I'd like
the wrapper function to give the same name)

  

def foo():


... return 7
...
  

foo.__name__ = 'bar'
foo(123)


Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() takes no arguments (1 given)

Thanks,
Andrew
  

In [9]: def foo():
  ...: return 7
  ...:

In [10]: bar = foo

In [11]: bar(54)

TypeError: foo() takes no arguments (1 given)

JM
--
http://mail.python.org/mailman/listinfo/python-list

Function name unchanged in error message

[andrew cooke]

Is there any way to change the name of the function in an error
message?  In the example below I'd like the error to refer to bar(),
for example (the motivation is related function decorators - I'd like
the wrapper function to give the same name)

>>> def foo():
... return 7
...
>>> foo.__name__ = 'bar'
>>> foo(123)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() takes no arguments (1 given)

Thanks,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a program

[Johann Spies]

On Fri, Jan 29, 2010 at 10:04:33AM +, Steven D'Aprano wrote:
> > I know this is a python list but if you really want to get the job done
> > quickly this is one method without writing python code:
> > 
> > $ cat /tmp/y
> > AGACTCGAGTGCGCGGA   0
> > AGATAAGCTAATTAAGCTACTGG 0
> > AGATAAGCTAATTAAGCTACTGGGTT   1
> > AGCTCACAATAT 1
> > AGGTCGCCTGACGGCTGC  0
> > $ grep -v 0 /tmp/y > tmp/z
> > $ cat /tmp/z
> > AGATAAGCTAATTAAGCTACTGGGTT   1
> > AGCTCACAATAT 1
> 
> That will do the wrong thing for lines like:
> 
> AGATAAGCTAATTAAGCTACTGGGTT   10

In that case change the grep to ' 0$'  then only the lines with a
singel digit '0' at the end of the line will be excluded.

One can do the same using regulare expressions in Python and it will
probably a lot slower on large files.

Regards
Johann
-- 
Johann Spies  Telefoon: 021-808 4599
Informasietegnologie, Universiteit van Stellenbosch

 "My son, if sinners entice thee, consent thou not."
Proverbs 1:10 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Anssi Saari]

Daniel Fetchinson  writes:

>> 1.  Python 3 is supported by major Linux distributions.
>>
>>  FALSE - most distros are shipping with Python 2.4, or 2.5 at best.
>
> This latter statement is false, Fedora 11 and 12 come with python 2.6.

How does your mention of one distro counter that claim? Personally,
I'd like to see a study of what version of Python ships with what
Linux distribution. Say, include the top 100 distros from
distrowatch.com? I think there may a surprising number of distros that
ship with no version of Python what so ever.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: myths about python 3

[Anssi Saari]

Stefan Behnel  writes:

> 'Stable Debian' has a long tradition of being late and outdated on arrival.
> That doesn't mean you can't use existing Debian packages on it.

Yes, but that's beside the point. No released version of Debian ships
with Python3 or even 2.6.

Oh, and RHEL5 and CentOS5 ship with 2.4.

-- 
http://mail.python.org/mailman/listinfo/python-list

[OT] myths about python 3

[Antoine Pitrou]

Le Fri, 29 Jan 2010 13:16:11 +1100, Ben Finney a écrit :
> 
> I think the reason “date” was initially used is because dates are most
> familiar to us as fleshy, dark brown, wrinkled, compressed points.
> 
> My interests in etymology and scatology unite here.

Ah, I suppose it explains the strange ASCII art drawing in your signature 
then?

 (Here)
   |
   |
   v

> --
>  \  “In the long run, the utility of all non-Free software |
>   `\  approaches zero. All non-Free software is a dead end.” —Mark |
> _o__)Pilgrim, 2006 |
> Ben Finney


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Perl 6 [was Re: myths about python 3]

[Grant Edwards]

On 2010-01-29, Neil Hodgson  wrote:

> Looks to me like the problem with Perl 6 was that it was too
> ambitious, wanting to fix all perceived problems with the
> language.

I thought Python was Perl with all the perceived problems fixed.

-- 
Grant

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 3's adoption

[Steven D'Aprano]

On Fri, 29 Jan 2010 07:10:01 +0100, Alf P. Steinbach wrote:

>>>> L = ["æ", "ø", "å"]   # This is in SORTED ORDER in Norwegian L

[...]

>>>> L.sort( key = locale.strxfrm )
>>>> L
>['å', 'æ', 'ø']
>>>> locale.strcoll( "å", "æ" )
>1
>>>> locale.strcoll( "æ", "ø" )
>-1
> 
> Note that strcoll correctly orders the strings as ["æ", "ø", "å"], that
> is, it would have if it could have been used as cmp function to sort (or
> better, to a separate routine named e.g. custom_sort).

This is in Python2.5, so I'm explicitly specifying unicode strings:

>>> L = [u"æ", u"ø", u"å"]
>>> assert sorted(L) == [u'å', u'æ', u'ø']

The default C-locale sorting of L does not equal to L. Now let's change 
to Norwegian locale:

>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'nb_NO')
'nb_NO'
>>> print u''.join(sorted(L, cmp=locale.strcoll))
æøå

So far so good, we've confirmed that in Python 2.5 we can sort in a 
locale-aware form. Now, can we do this with a key function? Thanks to 
Raymond Hettinger's recipe here:

http://code.activestate.com/recipes/576653/


>>> print u''.join(sorted(L, key=CmpToKey(locale.strcoll)))
æøå


Success!

Let's try it in Python 3.1:

>>> L = ["æ", "ø", "å"]
>>> assert sorted(L) == ['å', 'æ', 'ø']
>>>
>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'nb_NO')
'nb_NO'
>>> ''.join(sorted(L, key=CmpToKey(locale.strcoll)))
'æøå'


The definition of CmpToKey can be found at the URL above. It's not very 
exciting, but here it is:


def CmpToKey(mycmp):
'Convert a cmp= function into a key= function'
class K(object):
def __init__(self, obj, *args):
self.obj = obj
def __lt__(self, other):
return mycmp(self.obj, other.obj) == -1
def __gt__(self, other):
return mycmp(self.obj, other.obj) == 1
def __eq__(self, other):
return mycmp(self.obj, other.obj) == 0
def __le__(self, other):
return mycmp(self.obj, other.obj) != 1  
def __ge__(self, other):
return mycmp(self.obj, other.obj) != -1
def __ne__(self, other):
return mycmp(self.obj, other.obj) != 0
return K


If that's too verbose for you, stick this as a helper function in your 
application:


def CmpToKey(mycmp):
'Convert a cmp= function into a key= function'
class K(object):
def __init__(self, obj, *args):
self.obj = obj
__lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1
__gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1
__eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0
__le__ = lambda s, o: mycmp(s.obj, o.obj) != 1
__ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1
__ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0
return K


[...]
> The above may just be a bug in the 3.x stxfrm. But it illustrates that
> sometimes you have your sort order defined by a comparision function.
> Transforming that into a key can be practically impossible (it can also
> be quite inefficient).

This might be true, but you haven't demonstrated it. With one little 
helper function, you should be able to convert any comparison function 
into a key function, with relatively little overhead.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: which one is faster?

[Antoine Pitrou]

Le Thu, 28 Jan 2010 22:39:32 -0800, alex23 a écrit :
> On Jan 29, 4:29 pm, "Stephen.Wu" <54wut...@gmail.com> wrote:
>> str.find(targetStr)
>> str.index(targetStr) with exception
>> str.count(targetStr)
>> targetStr in str
>>
>> which is the fastest way to check whether targetStr is in str?
> 
[...]
> 
>>From the looks of those results, using 'in' seems to be the fastest.

To answer the question more precisely:

* "in" is the fastest because it doesn't have the method call overhead. 
This is only a fixed cost, though, and doesn't depend on the inputs.
* all four alternatives use the same underlying algorithm, *but* count() 
has to go to the end of the input string in order to count all 
occurrences. The other expressions can stop as soon as the first 
occurence is found, which can of course be a big win if the occurrence is 
near the start of the string and the string is very long

So, to sum it up:
* "in" is faster by a small fixed cost advantage
* "find" and "index" are almost exactly equivalent
* "count" will often be slower because it can't early exit

Regards

Antoine.

-- 
http://mail.python.org/mailman/listinfo/python-list

Thread get off from CPU occasionally. I suspect.

[Rajat]

Hi,

I'm using Python 2.6 and using the threading module.
In a python module (parent module) I'm importing main() function from
another python module. I'm runnin this main() function on a Python
thread so that I can periodically check in the parent thread/module if
the new thread is up and running. This information that the python
script running on the new thread is still up and running is passed to
a 3rd party application (in a manner known to 3rd party).

As 3rd party is waiting for this new thread to finish. It waits for 3
mins and on told to it by the parent thread that new thread is still
running, the 3rd party then starts to wait again for 3 mins.

I've successfully implemented this functionality and it runs well
except for a rare chance when the new thread does not completely runs.
However, most of the cases the new thread runs completely. I cannot
understand why and when the new thread gets off the CPU or crashes or
any other thing?

I'm not able to figure out what is happening.?

The code in the parent thread that starts the new thread is as
follows?

exec("from " + passed_script_file + " import main") #imported main
function. It works.
import threading
import time
p = RunTestScript(main)
p.start()
t_nought = time.time()
logSystemInfo(t_nought)
seconds_passed = 0
keep_alive_set = 0
while p.isAlive() == True:
p.join(60.0)
if p.isAlive() == True:
seconds_passed = time.time() - t_nought
if seconds_passed >= 120:
t_nought =  time.time()
if keep_alive_set != 1:# 3rd party way of informing
that the new thread is alive.
request = 'SET SHARED VAR KEEP_ALIVE=%d' %(1)
res = handle.submit('local', 'VAR', request)
if (res.rc != 0):
logSystemError('[PyInt] Error: STAF local VAR %s
failed, RC: %s, Result: %s\n' % (request, res.rc,
res.result))
return res.rc
keep_alive_set = 1



Earlier, when I did not run the main function on a new thread, it
always used to run completely. I used to run it as:

exec("from " + passed_script_file + " import main")
retCode = main()



Thanks and regards,
Rajat
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: which one is faster?

[Bruno Desthuilliers]

alex23 a écrit :

On Jan 29, 4:29 pm, "Stephen.Wu" <54wut...@gmail.com> wrote:

str.find(targetStr)
str.index(targetStr) with exception
str.count(targetStr)
targetStr in str

which is the fastest way to check whether targetStr is in str?


It's generally a lot quicker to investigate this kind of question
yourself using the interpreter & the timeit module. You'll need skills
like these to be able to profile your code to look for actual
performance bottlenecks, generic advice on the fastest of a set of
functions will only get you so far.


Indeed. Another point to take into consideration is the _real_ dataset 
on which the functions under test will have to work. And possibly the 
available resources at runtime. As an example, an iterator-based 
solution is likely to be slower than a more straightforward "load 
everything in ram" one, but will scale better for large datasets and/or 
heavy concurrent access to the system.



--
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a program

[Steven D'Aprano]

On Fri, 29 Jan 2010 11:23:54 +0200, Johann Spies wrote:

> On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote:
>> Hi folks,
>> 
>> I am a newbie to python, and I would be grateful if someone could point
>> out the mistake in my program. Basically, I have a huge text file
>> similar to the format below:
>> 
>> AGACTCGAGTGCGCGGA0
>> AGATAAGCTAATTAAGCTACTGG  0
>> AGATAAGCTAATTAAGCTACTGGGTT1
>> AGCTCACAATAT  1
>> AGGTCGCCTGACGGCTGC   0
> 
> I know this is a python list but if you really want to get the job done
> quickly this is one method without writing python code:
> 
> $ cat /tmp/y
> AGACTCGAGTGCGCGGA   0
> AGATAAGCTAATTAAGCTACTGG 0
> AGATAAGCTAATTAAGCTACTGGGTT   1
> AGCTCACAATAT 1
> AGGTCGCCTGACGGCTGC  0
> $ grep -v 0 /tmp/y > tmp/z
> $ cat /tmp/z
> AGATAAGCTAATTAAGCTACTGGGTT   1
> AGCTCACAATAT 1

That will do the wrong thing for lines like:

AGATAAGCTAATTAAGCTACTGGGTT   10


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a program

[Johann Spies]

On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote:
> Hi folks,
> 
> I am a newbie to python, and I would be grateful if someone could
> point out the mistake in my program. Basically, I have a huge text
> file similar to the format below:
> 
> AGACTCGAGTGCGCGGA 0
> AGATAAGCTAATTAAGCTACTGG   0
> AGATAAGCTAATTAAGCTACTGGGTT 1
> AGCTCACAATAT   1
> AGGTCGCCTGACGGCTGC0

I know this is a python list but if you really want to get the job
done quickly this is one method without writing python code:

$ cat /tmp/y
AGACTCGAGTGCGCGGA   0
AGATAAGCTAATTAAGCTACTGG 0
AGATAAGCTAATTAAGCTACTGGGTT   1
AGCTCACAATAT 1
AGGTCGCCTGACGGCTGC  0
$ grep -v 0 /tmp/y > tmp/z
$ cat /tmp/z
AGATAAGCTAATTAAGCTACTGGGTT   1
AGCTCACAATAT 1

Regards
Johann
-- 
Johann Spies  Telefoon: 021-808 4599
Informasietegnologie, Universiteit van Stellenbosch

 "My son, if sinners entice thee, consent thou not."
Proverbs 1:10 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError not caught by except statement

[siddhartha veedaluru]

Hi,

Thats the output i got in output window.it is not listing the statement
which caused it. copy paste and try it on you setup. it behaves the same
way.

I know that error is in log.info() function call.but it is not caught by
"except". which is not getting logged using log.exception

Thanks
Siddhartha


-- Forwarded message --
From: Steve Holden 
To:
Date: Tue, 26 Jan 2010 11:54:23 -0500
Subject: Re: TypeError not caught by except statement
siddhartha veedaluru wrote:
> Hi,
>
> except not able to caught the TypeError exception occured in the below
> code
>
>log.info 
> ("refer",ret)
in the try block
>
> throws a TypeError which is not caught .

The traceback you provide does not show the line in question, so I am
confused as to why you would think it was causing the problem. The
purpose of the traceback is to show you where the exception occurred,
and I question whether that is happening here.

Anyway, I suspect your error might go away if you turned the first
argument of hte log.info() call into a format string such as

 log.info("refer: %s", ret)

regards
 Steve

> Also sometimes process is getting hanged.
>
>

> import logging
> log = logging.getLogger()
> fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "-
> Log.log"
> log = logging.getLogger()
> log.setLevel(logging.NOTSET)
> fh = logging.FileHandler(logFile)
> logFileLevel = logging.DEBUG
> fh.setLevel(logFileLevel)
> format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s
> at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message)
> s"'
> fh.setFormatter(logging.Formatter(format_string))
> log.addHandler(fh)
>
> try:
>log.info ("start ")
>log.info ("refer",ret
)
>log.info ("end ")
> except TypeError:
>log.exception("Exception raised")
>
>
--
> OUTPUT message:
>
> Traceback (most recent call last):
>  File "C:\Python26\lib\logging\__init__.py", line 768, in emit
>msg = self.format(record)
>  File "C:\Python26\lib\logging\__init__.py", line 648, in format
>return fmt.format(record)
>  File "C:\Python26\lib\logging\__init__.py", line 436, in format
>record.message = record.getMessage()
>  File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage
>msg = msg % self.args
> TypeError: not all arguments converted during string formatting
>


--
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 76, Issue 303

[siddhartha veedaluru]

Hi,

Thats the output i got in output window.it is not listing the statement
which caused it. copy paste and try it on you setup. it behaves the same
way.

I know that error is in log.info() function call.but it is not caught by
"except". which is not getting logged using log.exception

Thanks
Siddhartha


-- Forwarded message --
From: Steve Holden 
To:
Date: Tue, 26 Jan 2010 11:54:23 -0500
Subject: Re: TypeError not caught by except statement
siddhartha veedaluru wrote:
> Hi,
>
> except not able to caught the TypeError exception occured in the below
> code
>
>log.info 
> ("refer",ret)
in the try block
>
> throws a TypeError which is not caught .

The traceback you provide does not show the line in question, so I am
confused as to why you would think it was causing the problem. The
purpose of the traceback is to show you where the exception occurred,
and I question whether that is happening here.

Anyway, I suspect your error might go away if you turned the first
argument of hte log.info() call into a format string such as

 log.info("refer: %s", ret)

regards
 Steve

> Also sometimes process is getting hanged.
>
>

> import logging
> log = logging.getLogger()
> fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "-
> Log.log"
> log = logging.getLogger()
> log.setLevel(logging.NOTSET)
> fh = logging.FileHandler(logFile)
> logFileLevel = logging.DEBUG
> fh.setLevel(logFileLevel)
> format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s
> at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message)
> s"'
> fh.setFormatter(logging.Formatter(format_string))
> log.addHandler(fh)
>
> try:
>log.info ("start ")
>log.info ("refer",ret
)
>log.info ("end ")
> except TypeError:
>log.exception("Exception raised")
>
>
--
> OUTPUT message:
>
> Traceback (most recent call last):
>  File "C:\Python26\lib\logging\__init__.py", line 768, in emit
>msg = self.format(record)
>  File "C:\Python26\lib\logging\__init__.py", line 648, in format
>return fmt.format(record)
>  File "C:\Python26\lib\logging\__init__.py", line 436, in format
>record.message = record.getMessage()
>  File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage
>msg = msg % self.args
> TypeError: not all arguments converted during string formatting
>


--
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wrap a function

[Joan Miller]

On 29 ene, 05:44, Dennis Lee Bieber  wrote:
> On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller
>  declaimed the following in
> gmane.comp.python.general:
>
> > On 28 ene, 19:16, Josh Holland  wrote:
> > > On 2010-01-28, Joan Miller  wrote:
>
> > > > I've to call to many functions with the format:
>
> > >  run("cmd")
>
> > > Check the docs on os.system().
> > No. I've a function that uses subprocess to run commands on the same
> > shell and so substitute to bash scrips. But a script full of run
> > ("shell_command --with --arguments") is too verbose.
>
>         I shall blaspheme, and suggest that maybe the language you want to
> use is REXX (ooREXX or Regina).
>
>         By default, ANY statement that can not be confused for a REXX
> language statement is sent to the currently defined command handler
> (Which on most OSs is equivalent to Python's os.system() call; the late
> Amiga, and IBM's mainframe OS had features that support defining other
> applications as command handlers).
>
>         A common practice is to put quotes about the first word of the
> command to ensure it gets treated as external command.
I prefer Python since that has a great standard library where I get a
well logging system between another things. In addition, the main
intended audience will be for system administrators who many of them
already have used it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting a list with entries of unequal types

[Peter Otten]

Ben Finney wrote:

> Peter Otten <__pete...@web.de> writes:
> 
>> I can't find the relevant part of the 2.6 documentation, but something
>> like
>>
>> >>> def key(x):
>> ... t = type(x)
>> ... t = compat.get(t, t)
>> ... return t.__name__, id(t), x
>> ...
>> >>> compat = {bool: float, int: float}
> 
> The problem with this approach is that it assumes I know all the types
> of elements that will be sorted; I don't. I want to sort a list coming
> from some other part of the code, and I don't want to arbitrarily limit
> the types of the list elements.

I don't see the limitation here. The above key functions sorts unknown types 
by name, then id(type) then instance.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list