Re: [Python-Dev] reference leaks, __del__, and annotations

[Neal Norwitz]

On 4/3/06, Michael Hudson <[EMAIL PROTECTED]> wrote:
> Greg Ewing <[EMAIL PROTECTED]> writes:
>
> > Michael Hudson wrote:
> >
> >> And if we want to have a version of __del__ that can't reference
> >> 'self', we have it already: weakrefs with callbacks.
> >
> > Does that actually work at the moment? Last I heard,
> > there was some issue with gc and weakref callbacks
> > as well. Has that been resolved?
>
> Talk about FUD.  Yes, it works, as far as I know.

Not sure if everyone is talking about the same thing.  This is still a
problem (at least for me): 
http://svn.python.org/projects/python/trunk/Lib/test/crashers/weakref_in_del.py

It creates a weakref to self in __del__.

There are 7 crashers, plus 5 more due to infinite recursion. :-(  That
doesn't include the parts of test_trace that are commented out.  At
least test_trace needs to be fixed prior to 2.5.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r43545 - in python/trunk: Doc/lib/libcalendar.tex Lib/calendar.py

[Walter Dörwald]

Greg Ewing wrote:

> Walter Dörwald wrote:
> 
>> OK, the property setter does a "% 7" now. (But the global
>> setfirstweekday() still does a range check).
> 
> Wouldn't it be better for the setter to raise an exception
> if it's out of range? It probably indicates a bug in the
> caller's code.

The day before Monday is -1, so it adds a little convenience. If this 
convenience is really worth it is a completely different topic.

I think we've spent more time discussing the calendar module, than the 
Python community has spent using it! ;)

Bye,
Walter Dörwald

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Renaming sqlite3

[Thomas Heller]

Martin v. Löwis wrote:
> Thomas Heller wrote:
>> But if you make the change to implement option 3, IMO it would be a
>>  good idea to add the Python version number to the .pyd basename as
>>  well.
> 
> Can you please elaborate? In the name of what .pyd file do you want
> the Python version number? And why?

Since on Windows binary extensions have to be compiled for the exact
major version of Python, it seems logical (to me, at least), to include
that version number into the filename.  Say, _socket25.pyd.

> And why is that related to not supporting extensions with .DLL names
> anymore?

IMO changing that would require a PEP (but I may be wrong), and this is only
another idea which should be considered when writing or discussing that.
If you don't like the idea, or don't see any advantages in this, I retract the
request.

>> pywin32 already has to do this, since pythoncomXY.pyd and 
>> pywintypesXY.pyd have to live (if possible) in the windows
>> directory.
> 
> I think this is a very special case: it could have been implemented 
> with separate DLLs which just provide the COM entry points, and find
> the location of pythoncom.pyd from the registry. I would discourage 
> people from providing additional entry points to an extension module.
> 
> 
> Regards, Martin ___ 
> Python-Dev mailing list Python-Dev@python.org 
> http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
> 
> 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Michael Hudson]

"Neal Norwitz" <[EMAIL PROTECTED]> writes:

> On 4/3/06, Michael Hudson <[EMAIL PROTECTED]> wrote:
>> Greg Ewing <[EMAIL PROTECTED]> writes:
>>
>> > Michael Hudson wrote:
>> >
>> >> And if we want to have a version of __del__ that can't reference
>> >> 'self', we have it already: weakrefs with callbacks.
>> >
>> > Does that actually work at the moment? Last I heard,
>> > there was some issue with gc and weakref callbacks
>> > as well. Has that been resolved?
>>
>> Talk about FUD.  Yes, it works, as far as I know.
>
> Not sure if everyone is talking about the same thing.  This is still a
> problem (at least for me): 
> http://svn.python.org/projects/python/trunk/Lib/test/crashers/weakref_in_del.py
>
> It creates a weakref to self in __del__.

Yes, but that has nothing to do with the cycle collector.  I even have
a way to fix it, but I don't know if it breaks anything else...

Cheers,
mwh

-- 
  I wouldn't trust the Anglo-Saxons for much anything else.  Given
  they way English is spelled, who could trust them on _anything_ that
  had to do with writing things down, anyway?
-- Erik Naggum, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use dlopen() on Darwin/OS X to load extensions?

[Zachary Pincus]

On Apr 4, 2006, at 1:29 AM, Martin v. Löwis wrote:

> Zachary Pincus wrote:
>> Specifically, this patch would change a core python code path.
>
> Why do you think so? I believe Python always passes absolute paths
> to dlopen, so any path resolution dlopen might do should be  
> irrelevant.
> *If* you can get dlopen to look at directories outside sys.path,
> that would be a serious problem. You can use ktrace to find out
> what places it looks at.

Sorry, I meant path in a more metaphoric sense; e.g. on OS X /  
Darwin, Python used to go through the code in dynload_next.c every  
time it loaded an extension, now under the proposed patch it goes  
through dynload_shlib.c.

>
>> But it would be good to have a specific benchmark to know nothing
>> will break. I personally sort of feel that if dlopen() works once or
>> twice, it will probably always work, but there are those who probably
>> understand better the failure modes of opening shared libs as python
>> extensions, and could suggest some good things to test.
>
> Running the test suite should already exercise this code a lot.
> You should run the test suite both in "working copy" mode, and
> in "make install" mode; if you know how to produce a Mac installer,
> testing it in such an installer ("framework mode"?) could also
> be done.

I'll do this, thanks.

Zach

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] tally (and other accumulators)

[Alex Martelli]

It's a bit late for 2.5, of course, but, I thought I'd propose it  
anyway -- I noticed it on c.l.py.

In 2.3/2.4 we have many ways to generate and process iterators but  
few "accumulators" -- functions that accept an iterable and produce  
some kind of "summary result" from it.  sum, min, max, for example.  
And any, all in 2.5.

The proposed function tally accepts an iterable whose items are  
hashable and returns a dict mapping each item to its count (number of  
times it appears).

This is quite general and simple at the same time: for example, it  
was proposed originally to answer some complaint about any and all  
giving no indication of the count of true/false items:

tally(bool(x) for x in seq)

would give a dict with two entries, counts of true and false items.

Just like the other accumulators mentioned above, tally is simple to  
implement, especially with the new collections.defaultdict:

import collections
def tally(seq):
 d = collections.defaultdict(int)
 for item in seq:
 d[item] += 1
 return dict(d)

Nevertheless, simplicity and generality make it advisable to supply  
it as part of the standard library (location TBD).

A good alternative would be a classmethod tally within  
collections.defaultdict, building and returning a defaultdict as  
above (with a .factory left to int, for further possible use as a  
'bag'/multiset data structure); this would solve the problem of where  
to locate tally if it were to be a function.  defaultdict.tally would  
be logically quite similar to dict.fromkeys, except that keys  
happening repeatedly get counted (and so each associated to a value  
of 1 and upwards) rather than "collapsed".


Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Jeremy Hylton]

On 4/4/06, Alex Martelli <[EMAIL PROTECTED]> wrote:
> import collections
> def tally(seq):
>  d = collections.defaultdict(int)
>  for item in seq:
>  d[item] += 1
>  return dict(d)
>
> Nevertheless, simplicity and generality make it advisable to supply
> it as part of the standard library (location TBD).
>
> A good alternative would be a classmethod tally within
> collections.defaultdict, building and returning a defaultdict as
> above (with a .factory left to int, for further possible use as a
> 'bag'/multiset data structure); this would solve the problem of where
> to locate tally if it were to be a function.  defaultdict.tally would
> be logically quite similar to dict.fromkeys, except that keys
> happening repeatedly get counted (and so each associated to a value
> of 1 and upwards) rather than "collapsed".

Putting it somewhere in collections seems like a great idea. 
defaultdict is a bit odd, because the functionality doesn't have
anything to do with defaults, just dicts.  maybe a classmethod on
regular dicts would make more sense?

I write this function regularly, so I'd be happy to have it available directly.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Alex Martelli]

On Apr 4, 2006, at 8:01 AM, Jeremy Hylton wrote:

> On 4/4/06, Alex Martelli <[EMAIL PROTECTED]> wrote:
>> import collections
>> def tally(seq):
>>  d = collections.defaultdict(int)
>>  for item in seq:
>>  d[item] += 1
>>  return dict(d)
...
> Putting it somewhere in collections seems like a great idea.
> defaultdict is a bit odd, because the functionality doesn't have
> anything to do with defaults, just dicts.  maybe a classmethod on
> regular dicts would make more sense?

Good points: it should probably be a classmethod on dict, or a  
function in module collections.

> I write this function regularly, so I'd be happy to have it  
> available directly.

Heh, same here -- soon as I saw it proposed on c.l.py I recognized an  
old friend and it struck me that, simple but widely used, it should  
be somewhere in the standard library.


Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Tim Peters]

[Michael Hudson]
>>> And if we want to have a version of __del__ that can't reference
>>> 'self', we have it already: weakrefs with callbacks.

[Greg Ewing]
>> Does that actually work at the moment? Last I heard,
>> there was some issue with gc and weakref callbacks
>> as well. Has that been resolved?

[Michael]
> Talk about FUD.  Yes, it works, as far as I know.

I'm sure Greg has in mind this thread (which was in fact also the
thread that floated the idea of getting rid of __del__ in P3K):

http://mail.python.org/pipermail/python-dev/2004-November/049744.html

As that said, some weakref gc semantics are pretty arbitrary now, and
it gave two patches that implemented distinct semantic variants.  A
problem is that the variant semantics also seem pretty arbitrary ;-),
and there's a dearth of compelling use cases to guide a decision.  If
someone devoted enough time to seriously trying to get rid of __del__,
I suspect compelling use cases would arise.  I never use __del__
anyway, so my motivation to spend time on it is hard to detect.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] r43613 - python/trunk/Doc/lib/libcsv.tex

[David Goodger]

[regarding the examples in the last section of libcsv.tex]

[Walter Dörwald]
> These classes will have problems with various non-charmap encodings. 
> (E.g. the writer will write multiple BOMS for UTF-16). IMHO it would be 
> better to use the new incremental codecs in these examples.

Could you fix it?  I'm not familiar with the incremental codecs.

-- 
David Goodger 



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] r43613 - python/trunk/Doc/lib/libcsv.tex

[Walter Dörwald]

David Goodger wrote:

> [regarding the examples in the last section of libcsv.tex]
> 
> [Walter Dörwald]
>> These classes will have problems with various non-charmap encodings. 
>> (E.g. the writer will write multiple BOMS for UTF-16). IMHO it would be 
>> better to use the new incremental codecs in these examples.
> 
> Could you fix it?  I'm not familiar with the incremental codecs.

Sure! (Fixing the writer is simple, fixing the reader takes a bit more code)

Bye,
   Walter Dörwald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Thomas Wouters]

On 4/4/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
This apparently was due to a change in the behavior of __import__: in Python 2.4, __import__('') raises a ValueError; in Python 2.5, it returns None.Oops, that wasn't intended. I seem to recall seeing that before, and I thought I'd fixed it, but apparently I'm more senile than I thought. It's fixed in the trunk (right before alpha1 deadline ;)
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Ian Bicking]

Alex Martelli wrote:
> It's a bit late for 2.5, of course, but, I thought I'd propose it  
> anyway -- I noticed it on c.l.py.
> 
> In 2.3/2.4 we have many ways to generate and process iterators but  
> few "accumulators" -- functions that accept an iterable and produce  
> some kind of "summary result" from it.  sum, min, max, for example.  
> And any, all in 2.5.
> 
> The proposed function tally accepts an iterable whose items are  
> hashable and returns a dict mapping each item to its count (number of  
> times it appears).
> 
> This is quite general and simple at the same time: for example, it  
> was proposed originally to answer some complaint about any and all  
> giving no indication of the count of true/false items:
> 
> tally(bool(x) for x in seq)
> 
> would give a dict with two entries, counts of true and false items.
> 
> Just like the other accumulators mentioned above, tally is simple to  
> implement, especially with the new collections.defaultdict:
> 
> import collections
> def tally(seq):
>  d = collections.defaultdict(int)
>  for item in seq:
>  d[item] += 1
>  return dict(d)

Or:

   import collections
   bag = collections.Bag([1, 2, 3, 2, 1])
   assert bag.count(1) == 2
   assert bag.count(0) == 0
   assert 3 in bag
   # etc...


-- 
Ian Bicking  /  [EMAIL PROTECTED]  /  http://blog.ianbicking.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use dlopen() on Darwin/OS X to load extensions?

[Bob Ippolito]

On Apr 3, 2006, at 9:01 PM, Neal Norwitz wrote:

> On 4/3/06, Zachary Pincus <[EMAIL PROTECTED]> wrote:
>>
>> Sorry if it's bad form to ask about patches one has submitted -- let
>> me know if that sort of discussion should be kept strictly on the
>> patch tracker.
>
> No, it's fine.  Thanks for reminding us about this issue.
> Unfortunately, without an explicit ok from one of the Mac maintainers,
> I don't want to add this myself.  If you can get Bob, Ronald, or Jack
> to say ok, I will apply the patch ASAP.  I have a Mac OS X.4 box and
> can test it, but don't know the suitability of the patch.

The patch has my OK (I gave it a while ago on pythonmac-sig).

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] r43613 - python/trunk/Doc/lib/libcsv.tex

[Walter Dörwald]

David Goodger wrote:
> [regarding the examples in the last section of libcsv.tex]
> 
> [Walter Dörwald]
>> These classes will have problems with various non-charmap encodings. 
>> (E.g. the writer will write multiple BOMS for UTF-16). IMHO it would be 
>> better to use the new incremental codecs in these examples.
> 
> Could you fix it?  I'm not familiar with the incremental codecs.

Done (in r43642). The new classes might be even useful as real code in
the stdlib. Of course this isn't full Unicode support for csv yet,
because the delimiter, quote and lineterminator characters can't be
Unicode characters yet.

Bye,
   Walter Dörwald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Georg Brandl]

Alex Martelli wrote:
> On Apr 4, 2006, at 8:01 AM, Jeremy Hylton wrote:
> 
>> On 4/4/06, Alex Martelli <[EMAIL PROTECTED]> wrote:
>>> import collections
>>> def tally(seq):
>>>  d = collections.defaultdict(int)
>>>  for item in seq:
>>>  d[item] += 1
>>>  return dict(d)
> ...
>> Putting it somewhere in collections seems like a great idea.
>> defaultdict is a bit odd, because the functionality doesn't have
>> anything to do with defaults, just dicts.  maybe a classmethod on
>> regular dicts would make more sense?
> 
> Good points: it should probably be a classmethod on dict, or a  
> function in module collections.
> 
>> I write this function regularly, so I'd be happy to have it  
>> available directly.
> 
> Heh, same here -- soon as I saw it proposed on c.l.py I recognized an  
> old friend and it struck me that, simple but widely used, it should  
> be somewhere in the standard library.

Why not make it collections.bag, like the following:


class bag(dict):
def __init__(self, iterable=None):
dict.__init__(self)
if iterable: self.update(iterable)

def update(self, iterable):
for item in iterable:
self.add(item)

def add(self, item):
self[item] = self.get(item, 0) + 1

def remove(self, item):
if self[item] == 1:
del self[item]
else:
self[item] -= 1

def count(self, item):
return self[item]


(etc.)

Georg












___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Thomas Wouters]

On 4/4/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
Of course anyone who is interested can run the Twisted test suite very easily and take a look at the failures themselves (if you have Twisted installed, "trial twisted" will do it) and can guess which errors/failures are specific to Python 
2.5 (for instance, the lack of PyCrypto in my 2.5-alpha install generates a lot of failures.) My AMD64 machine was giving a _lot_ of errors on zip(xrange(sys.maxint), iterable), which I now fixed in trunk. I'm re-running the tests to find out which ones are left over, but I'm having a hard time figuring out what to look at in trial's rather verbose logfile. There seem to be quite a few tracebacks involving Exception subclasses, in any case. Perhaps changing Exception's type in 
2.5 wasn't a good idea after all (but hey, that's what alphas are for ;)Oh, goodie, a segmentation fault. Let's see if I can reproduce it ;P-- Thomas Wouters <
[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Thomas Wouters]

On 4/4/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
Oh, goodie, a segmentation fault. Let's see if I can reproduce it ;PAnd so I could.  test_banana.CananaTestCase.testCrashNegativeLong crashes, because it calls PyString_AsStringAndSize() with an int-ptr as second argument (an adjacent ptr variable becomes garbage.) That's certainly a problem with the Py_ssize_t change. Martin, aren't all output variables (or ptr-variables, rather) supposed to be controlled by the 'PY_SSIZE_T_CLEAN' #define? People aren't going to notice their compiler warnings; I know I didn't :)
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Neil Schemenauer]

Greg Ewing <[EMAIL PROTECTED]> wrote:
> Okay, so would it be possible for a generator that
> needs finalisation to set up a weakref callback, suitably
> rooted somewhere so that the callback is reachable,
> that references enough stuff to clean up after the
> generator, without referencing the generator itself?

I think so but it depends on what the finalizer needs to reference.
If it references into the cycle then we are back to the same
problem.  I think you could also do the same thing with a sub-object
that has the __del__ method.  Maybe the generator would only create
the sub-object if it needed to perform finalization.

  Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Neal Norwitz]

On 4/4/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>
>
> On 4/4/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
> >
> >
> > Oh, goodie, a segmentation fault. Let's see if I can reproduce it ;P
>
>
> And so I could.
> test_banana.CananaTestCase.testCrashNegativeLong crashes,
> because it calls PyString_AsStringAndSize() with an int-ptr as second
> argument (an adjacent ptr variable becomes garbage.) That's certainly a
> problem with the Py_ssize_t change. Martin, aren't all output variables (or
> ptr-variables, rather) supposed to be controlled by the 'PY_SSIZE_T_CLEAN'
> #define? People aren't going to notice their compiler warnings; I know I
> didn't :)

I don't think so.  I think that macro controls Py_BuildValue etc (the
APIs in modsupport.h)  I don't think it controls anything else.  So
what that means is that Py_BuildValue("l") means something different
depending on ssize_t or not.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Renaming sqlite3

[Martin v. Löwis]

Thomas Heller wrote:
> Since on Windows binary extensions have to be compiled for the exact
> major version of Python, it seems logical (to me, at least), to include
> that version number into the filename.  Say, _socket25.pyd.

This would be a major change; it might break the build process of many
existing extensions.

Also, I fail to see the point: since _socket.pyd for Python 2.5 is in
a directory that is only on the Python 2.5 sys.path, it is unlikely
that the files are ever confused for belonging to a different version.

>> And why is that related to not supporting extensions with .DLL names
>> anymore?
> 
> IMO changing that would require a PEP (but I may be wrong), and this is only
> another idea which should be considered when writing or discussing that.
> If you don't like the idea, or don't see any advantages in this, I retract the
> request.

Ok - it's not that I dislike it. It would be technically feasible.
It would be some effort to implement. However, I would expect that
other people object more strongly.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Raymond Hettinger]

[Alex] 
> This is quite general and simple at the same time: for example, it  
> was proposed originally to answer some complaint about any and all  
> giving no indication of the count of true/false items:
> 
> tally(bool(x) for x in seq)
> 
> would give a dict with two entries, counts of true and false items.

FWIW, sum() works nicely for counting true entries:

>>> sum(x%3==0 for x in range(100))
   34


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Martin v. Löwis]

Thomas Wouters wrote:
> And so I could.  test_banana.CananaTestCase.testCrashNegativeLong
> crashes, because it calls PyString_AsStringAndSize() with an int-ptr as
> second argument (an adjacent ptr variable becomes garbage.) That's
> certainly a problem with the Py_ssize_t change. Martin, aren't all
> output variables (or ptr-variables, rather) supposed to be controlled by
> the 'PY_SSIZE_T_CLEAN' #define? People aren't going to notice their
> compiler warnings; I know I didn't :)

No: this is the discussion I had with MAL. You have to watch for
compiler warnings talking about incorrect pointer types. These
days, you can apply Fredrik's checker to find out that you are
using functions that output Py_ssize_t.

Ignoring the warning might cause crashes on 64-bit machines. On
32-bit machines, there should be any negative consequence.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Martin v. Löwis]

Neal Norwitz wrote:
> I don't think so.  I think that macro controls Py_BuildValue etc (the
> APIs in modsupport.h)  I don't think it controls anything else.  So
> what that means is that Py_BuildValue("l") means something different
> depending on ssize_t or not.

Actually, it *only* affects s#, z#, t#. "l" will always go along with
a "long int" argument.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Brett Cannon]

On 4/4/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>
>
> On 4/4/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> >
> > Of course anyone who is interested can run the Twisted test suite very
> easily and take a look at the failures themselves (if you have Twisted
> installed, "trial twisted" will do it).
>
>
> ... and can guess which errors/failures are specific to Python 2.5 (for
> instance, the lack of PyCrypto in my 2.5-alpha install generates a lot of
> failures.) My AMD64 machine was giving a _lot_ of errors on
> zip(xrange(sys.maxint), iterable), which I now fixed in trunk. I'm
> re-running the tests to find out which ones are left over, but I'm having a
> hard time figuring out what to look at in trial's rather verbose logfile.
> There seem to be quite a few tracebacks involving Exception subclasses, in
> any case. Perhaps changing Exception's type in 2.5 wasn't a good idea after
> all (but hey, that's what alphas are for ;)

No, the idea was a wonderful idea!  =)

Are the errors because of something in Python, or because Twisted has
not been changed to handle the new semantics?  I know so far test
failures have come from people making overly strict assumptions about
the output (e.g., I think it was Thomas or Greg who was getting
failures because ``type(Exception)`` outputted a different string for
doctest stuff).  Is that the case here?

-Brett

>
> Oh, goodie, a segmentation fault. Let's see if I can reproduce it ;P
>
>
> --
> Thomas Wouters < [EMAIL PROTECTED]>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me
> spread!
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] line numbers, pass statements, implicit returns

[Jeremy Hylton]

On 4/2/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> On 4/1/06, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> > There are several test cases in test_trace that are commented out.  We
> > did this when we merged the ast-branch and promised to come back to
> > them.  I'm coming back to them now, but the test aren't documented
> > well and the feature they test isn't specified well.
> >
> > The failing tests I've looked at so far involving pass statements and
> > implicit "return None" statements generated by the compiler.  The
> > tests seem to verify that
> >
> > 1) if you have a function that ends with an if/else where the body
> > of the else is pass,
> > there is no line number associated with the return
> > 2) if you have a function that ends with a try/except where the
> > body of the except is pass,
> > there is a line number associated with the return.
> >
> > Here's a failing example
> >
> > def ireturn_example():
> > a = 5
> > b = 5
> > if a == b:
> > b = a+1
> > else:
> > pass
> >
> > The code is traced and test_trace verifies that the return is
> > associated with line 4!
> >
> > In these cases, the ast compiler will always associate a line number
> > with the return.  (Technically with the LOAD_CONST preceding the
> > RETURN_VALUE.)  This happens pretty much be accident.  It always
> > associates a line number with the first opcode generated after a new
> > statement is visited.  Since a Pass statement node has no opcode, the
> > return generates the first opcode.
> >
> > Now I could add some special cases to the compiler to preserve the old
> > behavior, the question is: Why bother?  It's such an unlikely case (an
> > else that has no effect).  Does anyone depend on the current behavior
> > for the ireturn_example()?  It seems sensible to me to always generate
> > a line number for the pass + return case, just so you see the control
> > flow as you step through the debugger.
>
> Makes sense to me. I can't imagine what this was testing except
> perhaps a corner case in the algorithm for generating the (insanely
> complicated) linenumber mapping table.
>
> > The other case that has changed is that the new compiler does not
> > generate code for "while 0:"  I don't remember why <0.5 wink>.  There
> > are several test cases that verify line numbers for code using this
> > kind of bogus construct.  There are no lines anymore, so I would
> > change the tests so that they don't expect the lines in question.  But
> > I have no idea what they are trying to test.  Does anyone know?
>
> Not me. This is definitely not part of the language spec! :-)

It needs to be part of some spec, I think.  It's probably part of the
implementation rather than the language, but we need some description
of what the trace hooks are supposed to do.  The unittests are some
kind of spec, but not a great one (because they don't seem exhaustive
and don't explain why they work the way they do).  Should we write an
informational PEP, then, that explains the conditions under which you
get a line event in trace mode?  And how much do we care about
compatibility from release to release?

Here's another case that came up while pursuing these tests.  It looks
like the old compiler generated lnotab entries with a 0 change in line
number, merely so that the trace code would generate an event on that
line.  The problem case is a for loop, where there is a line number
set for the SETUP_LOOP instruction.  You'd like a line event for the
last time through the for loop, when the iterator is consumed, and to
get that it appears that the lnotab has an entry with a 0 increment in
line number and a non-zero increment in bytecode offset.  There's no
comment in the lnotab code that mentions that as a possibility, so it
came as a surprise to me :-).  The question, then, is how many cases
are there like this.  It may only be possible to answer this by
walking through the old compiler code and noting all the uncommented
special cases.

Looks like a project for alpha 2.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] chilling svn for 2.5a1

[Neal Norwitz]

The freeze is still a few hours away, but please refrain from
modifications unless they are really, really important for the
release.  The tests are just starting to turn green again.

I'm not as worried about doc changes.  Andrew feel free to keep
updating whatsnew.

Cheers,
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[James Y Knight]

On Apr 3, 2006, at 4:02 PM, Tim Peters wrote:
> Using which compiler?  This varies across boxes.  Most obviously, on a
> 64-bit box all these members are 8 bytes (note that ob_refcnt is
> Py_ssize_t in 2.5, not int anymore), but even on some 32-bit boxes the
> "long double" trick only forces 4-byte alignment.

Hm, yes, my mistake. I see that on linux/x86, long double only takes  
12 bytes and is 4-byte aligned. Even though the actual CPU really  
wants it 8-byte aligned, the ABI has not been changed to allow that.

On OSX/ppc32, OSX/ppc64 and linux/x86-64, doubles are 16 bytes, and 8- 
byte aligned. So the struct uses 16 bytes or 32 bytes, and has the  
extra word of unused space.

All right, then, you could use the top bit of the ob_refcnt field.  
There's no way to possibly have 2**32 objects on a 32bit system anyhow.

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Thomas Wouters]

On 4/4/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
Thomas Wouters wrote:> And so I could.  test_banana.CananaTestCase.testCrashNegativeLong> crashes, because it calls PyString_AsStringAndSize() with an int-ptr as> second argument (an adjacent ptr variable becomes garbage.) That's
> certainly a problem with the Py_ssize_t change. Martin, aren't all> output variables (or ptr-variables, rather) supposed to be controlled by> the 'PY_SSIZE_T_CLEAN' #define? People aren't going to notice their
> compiler warnings; I know I didn't :)No: this is the discussion I had with MAL. You have to watch forcompiler warnings talking about incorrect pointer types. Thesedays, you can apply Fredrik's checker to find out that you are
using functions that output Py_ssize_t.Ignoring the warning might cause crashes on 64-bit machines. On32-bit machines, there should be any negative consequence.I assume you meant "shouldn't be any negative consequences" there ;) I thought this over during dinner (, curry) and I agree that we shouldn't make PY_SSIZE_T_CLEAN change output variables that the compiler can catch. Too much effort, and in the end it'll just cause extensions compiled for 64-bit python to not use 64-bit values 'silently' (although eventually something'll get truncated and people will get confused.) This at least gives a compile-time hint that, on 64-bit platforms, things aren't quite right.
It's just too bad that it's an easily overlooked hint, and that you can't get that hint when compiling for 32-bit platforms. Extension writers with 64-bit hardware access and the desire to compile, let alone test, on said hardware is still rare. If we did make PY_SSIZE_T_CLEAN adjust all Py_ssize_t*-using functions (which would mean a *lot* of API duplication), we could add a #warning for every use of the old API calls, so everyone sees it, even on 32-bit platforms. The warning should contain a pointer to a document describing how to conveniently support both Python 
2.5+ and 2.4-, though. Too much work for alpha1, in any case, and probably too much work period. The webpage should be made, though, if just to refer to in the release notes.-- Thomas Wouters <
[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Alex Martelli]

On 4/4/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Alex]
> > This is quite general and simple at the same time: for example, it
> > was proposed originally to answer some complaint about any and all
> > giving no indication of the count of true/false items:
> >
> > tally(bool(x) for x in seq)
> >
> > would give a dict with two entries, counts of true and false items.
>
> FWIW, sum() works nicely for counting true entries:
>
> >>> sum(x%3==0 for x in range(100))
>34

Sure, and also works fine for counting false ones, thanks to 'not',
but if you need both counts sum doesn't work (not without dirty tricks
that can't be recommended;-).


Alex
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Fredrik Lundh]

Thomas Wouters wrote:

> The webpage should be made, though, if just to refer to in the
> release notes.

the web page does exist:

http://www.python.org/dev/peps/pep-0353/#conversion-guidelines





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] outstanding items for 2.5

[Aahz]

On Mon, Apr 03, 2006, Guido van Rossum wrote:
>
> Done. What exactly do you plan to do apart from editing the docs to
> steer people away from file()?

For the initial checkin, the dirt-simple:

def open(filename, *args, **kwargs):
return file(filename, *args, **kwargs)

At this point, the sole purpose is to kill open() as a simple alias for
file() and turn it into a factory function that can be documented
separately (and have its own help() info).  Further extensions can be
done later.  Because this is file I/O, there is no reason to even think
of performance improvements in the call.
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"Look, it's your affair if you want to play with five people, but don't
go calling it doubles."  --John Cleese anticipates Usenet
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Twisted and Python 2.5a0r43587

[Martin v. Löwis]

Thomas Wouters wrote:
> It's just too bad that it's an easily overlooked hint, and that you
> can't get that hint when compiling for 32-bit platforms.

That all depends on the compiler, of course. Both MSVC and the Intel
compiler support a /Wp64 flag, which tells you about "questionable"
conversions between types even if the types currently happen to be
the same.

Python's PC/pyconfig.h defines Py_ssize_t (indirectly) as

typedef _W64 int ssize_t;

where _W64 expands to __w64, which clues the compiler that ssize_t
(and thus Py_ssize_t) would be 64 bits on Win64, and that truncation
and pointer type conflicts can arise when this is mixed with int
(resp. int*).

This is the second part (you can get the hint on a 32-bit system
if you use the right compiler).

On the first issue, I wish it was possible to tell the compiler
that this kind of pointer conversion is really an error.

For gcc, this is possible to do: -pedantic-errors changes all
warnings that are considered "pedantic" to errors. A pedantic warning
is a diagnostic that ISO C requires from a conforming implementation.
The specific problem (converting type pointers in parameter passing
without explicit cast) is such a required diagnostic, and thus
causes compilation to abort.

Unfortunately, this option is *really* pedantic. It first complains
about "long long", which can be overridden with -std=gnu99; it then
aborts at

Objects/methodobject.c: In function 'meth_hash':
Objects/methodobject.c:232: error: ISO C forbids conversion of function
pointer to object pointer type

This kind of conversion is frequent in Python (e.g. in typeobject.c),
so for Python itself, this is no easy option. To fix that, Python
would have to replace void* with, say, void(*)(void) in all places
where it passes function pointers around.

> Extension
> writers with 64-bit hardware access and the desire to compile, let alone
> test, on said hardware is still rare. If we did make PY_SSIZE_T_CLEAN
> adjust all Py_ssize_t*-using functions (which would mean a *lot* of API
> duplication), we could add a #warning for every use of the old API
> calls, so everyone sees it, even on 32-bit platforms.

How would you do that? You could trigger a linker warning on platforms
that support that, but that would be different from #warning - I doubt
you can cause a compiler warning if a certain function is called.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Martin v. Löwis]

James Y Knight wrote:
> All right, then, you could use the top bit of the ob_refcnt field.  
> There's no way to possibly have 2**32 objects on a 32bit system anyhow.

Of this kind, we have several spare bits: there are atleast two bits
in the ob_type field, since the type pointer is atleast 4-aligned
on all platforms we support.

OTOH, using these bits would break existing code.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] current 2.5 status

[Neal Norwitz]

Debian (ia64, ppc) is broken for several reasons, including the
following test failures:  test_ctypes test_curses test_sqlite.  Some
of the reasons vary depending on architecture.

Ubuntu on hppa seems to have problems with at least:  test_wait[34]
seems to crash.

cygwin has major problems (ie, it crashes) with test_bsddb3.  This may
actually be exposing a bug in bsddb.

None of these are show stoppers IMO.

Coverity didn't run last night.  The only outstanding issues were due
to sqlite.  Many, perhaps all, of these issues have been addressed.

The following systems are green:
Sparc Solaris 10
amd64 & x86 gentoo
PPC OS X.4
Alpha Tru64 5.1
x86 Windows 2k & XP (though there were some problems due to previous runs)
x86 OpenBSD

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] current 2.5 status

[Tim Peters]

[Neal Norwitz]
> ...
> The following systems are green:
> ...
> x86 Windows 2k & XP (though there were some problems due to previous runs)

There are no problems on one of the XP slaves.

test_sqlite never ran on Win2K, although it's unlikely it will fail
there (given that test_sqlite has run successfully on WinXP).

For all platforms, note that the buildbot currently doesn't test
release builds, or -O, at all.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] current 2.5 status

[Tim Peters]

Trent, FYI, on my box the invariable cause for

LINK : fatal error LNK1104: cannot open file './python25_d.dll'

in a failed buildbot Windows compile step is that some previous run
left behind a python_d.exe process that won't go away.  I don't know
why it won't go away on its own, or how it gets into that state, but I
do know that once it happens it never fixes itself.  You have to find
the python_d process(es) and kill it yourself.  Then the buildbot
dance works fine again.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] current 2.5 status

[Trent Mick]

[Tim Peters wrote]
> Trent, FYI, on my box the invariable cause for
> 
> LINK : fatal error LNK1104: cannot open file './python25_d.dll'
> 
> in a failed buildbot Windows compile step is that some previous run
> left behind a python_d.exe process that won't go away.  I don't know
> why it won't go away on its own, or how it gets into that state, but I
> do know that once it happens it never fixes itself.  You have to find
> the python_d process(es) and kill it yourself.  Then the buildbot
> dance works fine again.

In this case it was because python_d.exe crashed and Windows then popped
up a system modal dialog to tell me: hanging the process.

I wonder if it would be possible to write a "reaper" script that used
some combination of EnumWindows, SendKeys, the Performance Monitoring
APIs (Pdh* function) and some elbow grease to click these crash dialogs
away.

Trent

-- 
Trent Mick
[EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] current 2.5 status

[Martin v. Löwis]

Trent Mick wrote:
> I wonder if it would be possible to write a "reaper" script that used
> some combination of EnumWindows, SendKeys, the Performance Monitoring
> APIs (Pdh* function) and some elbow grease to click these crash dialogs
> away.

The buildbot invokes itself a "reaper" procedure if the current build
step doesn't produce any output for some period of time (I think 2000s).
This would be the place where actions should be added, IMO.

For clicking crash dialogs, I wonder whether running the processes in
their own window station (CreateWindowStation) would help: I would hope
that the dialogs get forcefully closed when the window station get
closed.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] current 2.5 status

[Delaney, Timothy (Tim)]

Trent Mick wrote:

> I wonder if it would be possible to write a "reaper" script that used
> some combination of EnumWindows, SendKeys, the Performance Monitoring
> APIs (Pdh* function) and some elbow grease to click these crash
> dialogs away.

I've found for this type of thing AutoIt
 works well. An example that might
be applicable:

I've got an HTML bandwidth monitor I wrote that retrieves data from my
router and my ISP using XMLHTTP. Sometimes it takes a long time to come
back and the script pops up a dialog with "A script is making IE run
slow" (or something to that effect - forget the exact error message).

I've got an AutoIt script which waits until that windows appears, closes
it, closes the bandwidth monitor, starts it back up, then goes back to
waiting. The script is about 10 lines long.

AutoIt scripts can also be saved (and redistributed) as a standalone app
(approx 120kB in size). So part of setting up a Windows buildbot slave
could be to add such a script to run at startup.

However, if there's some way of doing it without installing extra stuff
on the slave, that would be preferable.

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r43545 - in python/trunk: Doc/lib/libcalendar.tex Lib/calendar.py

[Greg Ewing]

Walter Dörwald wrote:
> Greg Ewing wrote:
> 
>> Wouldn't it be better for the setter to raise an exception
>> if it's out of range? It probably indicates a bug in the
>> caller's code.
> 
> The day before Monday is -1, so it adds a little convenience.

In that case, why doesn't the global function
allow the same convenience?

> I think we've spent more time discussing the calendar module, than the 
> Python community has spent using it! ;)

It makes a nice change from adaptation and generic
functions. :-)

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Greg Ewing]

Tim Peters wrote:
> A problem is that the variant semantics also seem pretty arbitrary ;-),
> and there's a dearth of compelling use cases to guide a decision.

At the time I think I suggested that it would be
reasonable if weakref callbacks were guaranteed to
be called as long as they weren't trash themselves
and they didn't reference any trash. From what you
said in a recent message, it sounds like that's
the way it currently works.

These semantics would be sufficient to be able to
use weakrefs to register finalisers, I think. You
keep a global list of weakrefs with callbacks that
call the finalizer and then remove the weakref
from the global list.

I'll put together a prototype some time and see if
it works. I actually have a use case at the moment --
a couple of types that need to release external
resources, and since they're part of a library
I'm distributing, I can't be sure people won't put
them in cycles.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Greg Ewing]

Neil Schemenauer wrote:

> I think so but it depends on what the finalizer needs to reference.

That's really what I'm asking -- what *does* a
generator finaliser need to reference? Or does
it depend on what the generator's code does?

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Greg Ewing]

James Y Knight wrote:

> All right, then, you could use the top bit of the ob_refcnt field.  
> There's no way to possibly have 2**32 objects on a 32bit system anyhow.

That would slow down every Py_INCREF and Py_DECREF,
which would need to be careful to exclude the
top bit from their operations.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Greg Ewing]

Martin v. Löwis wrote:

> Of this kind, we have several spare bits: there are atleast two bits
> in the ob_type field,

And that would require changing all code that dereferenced
the ob_type field! This would be much worse -- at least
Py_INCREF and Py_DECREF are macros...

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Should issubclass() be more like isinstance()?

[Crutcher Dunnavant]

While nocking together a framework today, I ran into the amazing
limitations of issubclass().

A) issubclass() throws a TypeError if the object being checked is not
a class, which seems very strange. It is a predicate, and lacking a
isclass() method, it should just return False.
B) issubclass() won't work on a list of classs, the way isinstance() does.

Is there any support for patches which fix A and/or B?

--
Crutcher Dunnavant <[EMAIL PROTECTED]>
littlelanguages.com
monket.samedi-studios.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[Greg Ewing]

Crutcher Dunnavant wrote:

> A) issubclass() throws a TypeError if the object being checked is not
> a class, which seems very strange.

If I ever pass a non-class to issubclass() it's almost
certainly a bug in my code, and I'd want to know about
it.

On the rare occasions when I don't want this, I'm
happy to write

   isinstance(c, type) and issubclass(c, d)

> B) issubclass() won't work on a list of classs,
 > the way isinstance() does.

That sounds more reasonable. I can't think of any
reason why it shouldn't work.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[Crutcher Dunnavant]

On 4/4/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Crutcher Dunnavant wrote:
>
> > A) issubclass() throws a TypeError if the object being checked is not
> > a class, which seems very strange.
>
> If I ever pass a non-class to issubclass() it's almost
> certainly a bug in my code, and I'd want to know about
> it.

Perhaps, but is it worth distorting a predicate?

> On the rare occasions when I don't want this, I'm
> happy to write
>
>isinstance(c, type) and issubclass(c, d)

This doesn't work, did you mean?
  isinstance(c, types.ClassType) and issubclass(c, d)


> > B) issubclass() won't work on a list of classs,
>  > the way isinstance() does.
>
> That sounds more reasonable. I can't think of any
> reason why it shouldn't work.
>
> --
> Greg
>


--
Crutcher Dunnavant <[EMAIL PROTECTED]>
littlelanguages.com
monket.samedi-studios.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Tim Peters]

[Tim Peters]
>> A problem is that the variant semantics also seem pretty arbitrary ;-),
>> and there's a dearth of compelling use cases to guide a decision.

[Greg Ewing]
|> At the time I think I suggested that it would be
> reasonable if weakref callbacks were guaranteed to
> be called as long as they weren't trash themselves
> and they didn't reference any trash. From what you
> said in a recent message, it sounds like that's
> the way it currently works.

Nope, and that was the point of the first message in the thread I
referenced.  The issues were explained well in that thread, so I don't
want to repeat it all here again.

> These semantics would be sufficient to be able to
> use weakrefs to register finalisers, I think. You
> keep a global list of weakrefs with callbacks that
> call the finalizer and then remove the weakref
> from the global list.
>
> I'll put together a prototype some time and see if
> it works. I actually have a use case at the moment --
> a couple of types that need to release external
> resources, and since they're part of a library
> I'm distributing, I can't be sure people won't put
> them in cycles.

Note that it's very easy to do this with __del__.  The trick is for
your type not to have a __del__ method itself, but to point to a
simple "cleanup object" with a __del__ method.  Give that "contained"
object references to the resources you want to close, and you're done.
 Because your "real" objects don't have __del__ methods, cycles
involving them can't inhibit gc.  The cleanup object's only purpose in
life is to close resources.  Like:

class _RealTypeResourceCleaner:
def __init__(self, *resources):
self.resources = resources

def __del__(self):
if self.resources is not None:
for r in self.resources:
r.close()
self.resources = None

# and typically no other methods are needed, or desirable, in
# this helper class

class RealType:
def __init__(*args):
...
# and then, e.g.,
self.cleaner = _ResourceCleaner(resource1, resource2)

... tons of stuff, but no __del__ 

That's the simple, general way to mix cycles and __del__ without problems.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Reminder: TRUNK FREEZE. 2.5a1, 00:00 UTC, Wednesday 5th of April.

[Anthony Baxter]

Just a reminder - the trunk is currently frozen for 2.5a1. Please 
don't check anything into it until the release is done. I'll send an 
update when this is good.

-- 
Anthony Baxter <[EMAIL PROTECTED]>
It's never too late to have a happy childhood.
--- Begin Message ---

Now that the bug day has been and gone, it's time to cut 2.5a1. Please 
consider the trunk FROZEN from 00:00 UTC/GMT on Wednesday the 5th of 
April. I'll post again when it's unfrozen. 

Please help in not making the release manager cry because the trunk is 
broken. Thanks,

Anthony

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/anthony%40interlink.com.au

--- End Message ---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Tim Peters]

[Greg Ewing]
> That's really what I'm asking -- what *does* a
> generator finaliser need to reference? Or does
> it depend on what the generator's code does?

A generator finalizer can execute arbitrary Python code.  Note that
even if that was limited (which it isn't -- there are no limitations)
to executing "pass", it wouldn't be exploitable, because the GIL can
be released during any Python code, allowing other threads to run
anything before the finalizer returns.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] The "Need for Speed" Sprint, Reykjavik, Iceland, May 21-28, 2006

[Steve Holden]

EWT LLC and CCP h.f., having specific commercial interests in
doing so, have decided to sponsor a sprint on the Python
programming language with specific short- and medium-term
acceleration goals.

The sprint is also intended to stimulate broad consideration of
the future of Python, and specifically to try and identify the
most promising routes to improved performance.

Selected members of the Python developer community will receive
direct  invitations, and their attendance will be sponsored, to
try to ensure the most effective possible result. In the spirit
of open source events the sprint is open to all.

The venue is the Grand Hotel, Reykjavik, http://www.grand.is/.

The following objectives are of interest to the sponsors.

* Improving the decimal module by implementing portions in C

* Investigate whether RPython offers sufficient speedup over
   the regular CPython interpreter to replace tailored C and
   C++ code in MMP gaming applications

* Implement an ordered dictionary in both C and Python

* Implement data-structure-specific algorithms, which rely
   heavily on certain data structures, as RPython

* Adding an iterator interface (similar to re.finditer) to
   the struct module

* Further refine the PyPy LLVM back end to improve general
   execution speeds

* Offer the PyPy team a sprint venue to continue their
   development work on Python implementations written in Python

* Create a string subtype that provides lazy slicing without
   copying

Sprinters are, as always, free to choose the aspects of Python
they choose to work on.

Flights should be booked to and from Keflavik airport (KEF).

There will be a mid-week “let your hair down” social event to
which all sprint participants are cordially invited.

In order to ensure that sufficient meeting space is available,
please notify Steve Holden of your intention to attend by emailing
[EMAIL PROTECTED]

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogspot.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] reference leaks, __del__, and annotations

[Tim Peters]

[various people debating how to steal a bit from an existing PyObject member]

Let's stop this.  It's a "bike shed" argument:  even if we had 1000
spare bits, it wouldn't do any good.  Having a bit to record whether
an object has been finalized doesn't solve any problem that's been
raised recently (it doesn't cure any problems with resurrection, for
example).  Even if we wanted it just to support a slow, dubious way to
collect trash cycles containing objects with __del__ methods,
gcmodule.c would need major rewriting to make that happen.  If someone
is interested enough to do that, fine, but it sure ain't me ;-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[Guido van Rossum]

On 4/4/06, Crutcher Dunnavant <[EMAIL PROTECTED]> wrote:
> On 4/4/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> > Crutcher Dunnavant wrote:
> >
> > > A) issubclass() throws a TypeError if the object being checked is not
> > > a class, which seems very strange.
> >
> > If I ever pass a non-class to issubclass() it's almost
> > certainly a bug in my code, and I'd want to know about
> > it.
>
> Perhaps, but is it worth distorting a predicate?

Certainly. In other languages this would be a compile-time error.

There's no rule that predicate cannot raise an exception.

If you're not sure whether something is a class or not, you should
first sniff it out for its class-ness before checking whether it's a
subclass of something. I recommend hasattr(x, "__bases__") which is
more likely to recognize classes than isinstance(x, type) -- the
latter only works for standard new-style classes.

> > On the rare occasions when I don't want this, I'm
> > happy to write
> >
> >isinstance(c, type) and issubclass(c, d)
>
> This doesn't work, did you mean?
>   isinstance(c, types.ClassType) and issubclass(c, d)
>
>
> > > B) issubclass() won't work on a list of classs,
> >  > the way isinstance() does.
> >
> > That sounds more reasonable. I can't think of any
> > reason why it shouldn't work.

Agreed.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[Crutcher Dunnavant]

On 4/4/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> On 4/4/06, Crutcher Dunnavant <[EMAIL PROTECTED]> wrote:
> > On 4/4/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> > > Crutcher Dunnavant wrote:
> > >
> > > > A) issubclass() throws a TypeError if the object being checked is not
> > > > a class, which seems very strange.
> > >
> > > If I ever pass a non-class to issubclass() it's almost
> > > certainly a bug in my code, and I'd want to know about
> > > it.
> >
> > Perhaps, but is it worth distorting a predicate?
>
> Certainly. In other languages this would be a compile-time error.

In other, statically typed languges. Someone hands me X, I want to
know if X is a subclass of Y. Yes or No. If X is not a class, then it
is not a subclass of Y, hence issubclass(X, Y) should return False.

> There's no rule that predicate cannot raise an exception.

No, but it makes many applications (such as using it as a test in list
comprehensions) difficult enough to not be worth it.

> If you're not sure whether something is a class or not, you should
> first sniff it out for its class-ness before checking whether it's a
> subclass of something. I recommend hasattr(x, "__bases__") which is
> more likely to recognize classes than isinstance(x, type) -- the
> latter only works for standard new-style classes.
>
> > > On the rare occasions when I don't want this, I'm
> > > happy to write
> > >
> > >isinstance(c, type) and issubclass(c, d)
> >
> > This doesn't work, did you mean?
> >   isinstance(c, types.ClassType) and issubclass(c, d)
> >
> >
> > > > B) issubclass() won't work on a list of classs,
> > >  > the way isinstance() does.
> > >
> > > That sounds more reasonable. I can't think of any
> > > reason why it shouldn't work.
>
> Agreed.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>


--
Crutcher Dunnavant <[EMAIL PROTECTED]>
littlelanguages.com
monket.samedi-studios.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[Alex Martelli]

On Apr 4, 2006, at 8:18 PM, Crutcher Dunnavant wrote:
...
>> There's no rule that predicate cannot raise an exception.
>
> No, but it makes many applications (such as using it as a test in list
> comprehensions) difficult enough to not be worth it.

IMHO, the solution to THAT very real problem is to supply a built-in  
function that catches some exceptions and maps them into suitable  
return values. Simplest would be something like:

def catch(excepts, default, f, *a, **k):
 try:
 return f(*a, **k)
 except excepts:
 return default

and then, the LC you're after is easy:

subints = [x for x in y if catch(TypeError, False, issubclass, x, int)]

though I'm sure we can get better syntax if we turn 'catch' into some  
kind of syntactic special form.  My point is that there are umpteen  
predicates one can write which would have to be distorted to ensure  
they can't raise -- better to let them raise if they must, and allow  
catching the expected exception(s), somewhat like this example.


Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[James Y Knight]

On Apr 4, 2006, at 11:41 PM, Alex Martelli wrote:
> IMHO, the solution to THAT very real problem is to supply a built-in
> function that catches some exceptions and maps them into suitable
> return values. Simplest would be something like:
[...]
> though I'm sure we can get better syntax if we turn 'catch' into some
> kind of syntactic special form.  My point is that there are umpteen
> predicates one can write which would have to be distorted to ensure
> they can't raise -- better to let them raise if they must, and allow
> catching the expected exception(s), somewhat like this example.

And while you're at it, how about some syntax to use try/finally and  
while inside expressions too.

That will complete the duplication of flow control syntax between  
statement and expression form.

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reminder: TRUNK FREEZE. 2.5a1, 00:00 UTC, Wednesday 5th of April.

[Fred L. Drake, Jr.]

On Tuesday 04 April 2006 22:34, Anthony Baxter wrote:
 > Just a reminder - the trunk is currently frozen for 2.5a1. Please
 > don't check anything into it until the release is done. I'll send an
 > update when this is good.

Documentation packages have been uploaded to the download area.


  -Fred

-- 
Fred L. Drake, Jr.   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Jess Austin]

Alex wrote: 
> import collections
> def tally(seq):
>  d = collections.defaultdict(int)
>  for item in seq:
>  d[item] += 1
>  return dict(d)

I'll stop lurking and submit the following:

def tally(seq):
return dict((group[0], len(tuple(group[1])))
for group in itertools.groupby(sorted(seq)))

In general itertools.groupby() seems like a very clean way to do this
sort of thing, whether you want to end up with a dict or not.  I'll go
so far as to suggest that the existence of groupby() obviates the
proposed tally().  Maybe I've just coded too much SQL and it has warped
my brain...

OTOH the latter definition of tally won't cope with iterables, and it
seems like O(nlogn) rather than O(n).

cheers,
Jess
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Alex Martelli]

On Apr 4, 2006, at 10:53 PM, Jess Austin wrote:

> Alex wrote:
>> import collections
>> def tally(seq):
>>  d = collections.defaultdict(int)
>>  for item in seq:
>>  d[item] += 1
>>  return dict(d)
>
> I'll stop lurking and submit the following:
>
> def tally(seq):
> return dict((group[0], len(tuple(group[1])))
> for group in itertools.groupby(sorted(seq)))
>
> In general itertools.groupby() seems like a very clean way to do this
> sort of thing, whether you want to end up with a dict or not.  I'll go
> so far as to suggest that the existence of groupby() obviates the
> proposed tally().  Maybe I've just coded too much SQL and it has  
> warped
> my brain...
>
> OTOH the latter definition of tally won't cope with iterables, and it
> seems like O(nlogn) rather than O(n).

It will cope with any iterable just fine (not sure why you think  
otherwise), but the major big-O impact seems to me to rule it out as  
a general solution.


Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] tally (and other accumulators)

[Jess Austin]

Alex wrote:
> On Apr 4, 2006, at 10:53 PM, Jess Austin wrote:
> > Alex wrote:
> >> import collections
> >> def tally(seq):
> >>  d = collections.defaultdict(int)
> >>  for item in seq:
> >>  d[item] += 1
> >>  return dict(d)
> >
> > I'll stop lurking and submit the following:
> >
> > def tally(seq):
> > return dict((group[0], len(tuple(group[1])))
> > for group in itertools.groupby(sorted(seq)))
> >
> > In general itertools.groupby() seems like a very clean way to do this
> > sort of thing, whether you want to end up with a dict or not.  I'll go
> > so far as to suggest that the existence of groupby() obviates the
> > proposed tally().  Maybe I've just coded too much SQL and it has  
> > warped my brain...
> >
> > OTOH the latter definition of tally won't cope with iterables, and it
> > seems like O(nlogn) rather than O(n).
> 
> It will cope with any iterable just fine (not sure why you think  
> otherwise), but the major big-O impact seems to me to rule it out as  
> a general solution.

You're right in that it won't raise an exception on an iterator, but the
sorted() means that it's much less memory efficient than your version
for iterators.  Another reason to avoid sorted() for this application,
besides the big-O.  Anyway, I still like groupby() for this sort of
thing, with the aforementioned caveats.  Functional code seems a little
clearer to me, although I realize that preference is not held
universally.

cheers,
Jess
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should issubclass() be more like isinstance()?

[Georg Brandl]

Crutcher Dunnavant wrote:
> While nocking together a framework today, I ran into the amazing
> limitations of issubclass().
> 
> A) issubclass() throws a TypeError if the object being checked is not
> a class, which seems very strange. It is a predicate, and lacking a
> isclass() method, it should just return False.
> B) issubclass() won't work on a list of classs, the way isinstance() does.
> 
> Is there any support for patches which fix A and/or B?

I don't think B is broken:

Python 2.4.2 [...]
>>> issubclass(int, 0)
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: issubclass() arg 2 must be a class or tuple of classes
>>> issubclass(int, (int, str))
True
>>> issubclass(str, (int, str))
True
>>> issubclass(dict, (int, str))
False
>>>

(both isinstance() and issubclass() don't work on a _list_ as second argument)

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com