Re: [Python-Dev] Package Management - thoughts from the peanut gallery
Guys, I have taken the commitment to lead these tasks and synchronize the people that are willing to help on this. We are working on several tasks and PEPS to make things happen since the summit. There's no public roadmap yet on when things will be done (because there's no 100% certitude yet on what shall be done). But that it will probably be too late to see it happen in 3.1. Python 2.7 will be our target. The tasks discussed so far are: - version definition (http://wiki.python.org/moin/DistutilsVersionFight) - egg.info standardification (PEP 376) - metadata enhancement (rewrite PEP 345) - static metadata definition work (*) - creation of a network of OS packager people - PyPI mirroring (PEP 381) Each one of this task has a leader, except the one with (*). I just got back from travelling, and I will reorganize http://wiki.python.org/moin/Distutils asap to it is up-to-date. If you want to work on one of this task or feel there's a new task you can start, please, join Distutils SIG or contact me, Regards Tarek On Fri, Apr 3, 2009 at 6:55 AM, Stephen J. Turnbull step...@xemacs.orgwrote: Chris Withers writes: Personally I feel all of the above are perfectly possible, and can't see anyone being left unhappy by them. I'm sure I've missed something then, otherwise why not make it happen? Labor shortage. We will need a PEP, the PEP will need a sample implementation, and a proponent. Who's gonna bell the cat? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UnicodeDecodeError bug in distutils
On Fri, Apr 3, 2009 at 2:25 AM, Ben Finney ben+pyt...@benfinney.id.au wrote: Phillip J. Eby p...@telecommunity.com writes: However, there's currently no standard, as far as I know, for what encoding the PKG-INFO file should use. Who would define such a standard? PEP 376 where we can explain that all files in egg-info should be in a specific encoding My vote goes for “default is UTF-8”. +1 Meanwhile, the 'register' command accepts Unicode, but is broken in handling it. […] how so ? Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
Alexandre Vassalotti alexandre at peadrop.com writes: I just noticed that the new io-c modules were merged in the py3k branch (I know, I am kind late on the news—blame school work). Anyway, I am just wondering if it would be a good idea to put the io-c modules in a sub-directory (like sqlite), instead of scattering them around in the Modules/ directory. Welcome back! I have no particular opinion on this. I suggest waiting for Benjamin's advice and following it :-) (unless the FLUFL wants to chime in) Benjamin-makes-boring-decisions-easy'ly yrs, Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
Thomas Wouters thomas at python.org writes: Pystone is pretty much a useless benchmark. If it measures anything, it's the speed of the bytecode dispatcher (and it doesn't measure it particularly well.) PyBench isn't any better, in my experience. I don't think pybench is useless. It gives a lot of performance data about crucial internal operations of the interpreter. It is of course very little real-world, but conversely makes you know immediately where a performance regression has happened. (by contrast, if you witness a regression in a high-level benchmark, you still have a lot of investigation to do to find out where exactly something bad happened) Perhaps someone should start maintaining a suite of benchmarks, high-level and low-level; we currently have them all scattered around (pybench, pystone, stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not to mention other third-party stuff that can be found in e.g. the Computer Language Shootout). I also know Gregory P. Smith had emitted the idea of plotting benchmark figures for each new revision of trunk or py3k (and, perhaps, other implementations), but I don't know if he's willing to do it himself :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's update CObject API so it is safe and regular!
Here's one from EVE, where the DB module creates raw data, for our Crowsets,
and then hands it over to another module for consumption (actual creation of
the CRow and CrowDescriptor objects:
BluePy raw(PyCObject_FromVoidPtr(mColumnList, 0));
if (!raw)
return 0;
return PyObject_CallMethod(blueModule, DBRowDescriptor, O, raw.o);
This is done for performance reasons to avoid data duplication. Of course it
implies tight coupling of the modules.
In our FreeType wrapper system, we also use it to wrap pointers to FreeType
structs:
template class T
struct Wrapper : public T
{
...
PyObject *Wrap() {if (!sMap.size())Init(); return
PyCObject_FromVoidPtrAndDesc(this, sMap, 0);}
};
It is quite useful to pass unknown and opaque stuff around with, really, and
makes certain things possible that otherwise wouldn't be.
We live with the type unsafety, of course.
In fact, I don't think we ever use a CObject to expose an API.
Kristj'an
-Original Message-
From: python-dev-bounces+kristjan=ccpgames@python.org
[mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Guido
van Rossum
Sent: 2. apríl 2009 17:19
To: Jim Fulton
Cc: Python-Dev@python.org
Subject: Re: [Python-Dev] Let's update CObject API so it is safe and regular!
On Thu, Apr 2, 2009 at 6:22 AM, Jim Fulton j...@zope.com wrote:
The original use case for CObjects was to export an API from a module, in
which case, you'd be importing the API from the module.
I consider this the *only* use case. What other use cases are there?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] sequence slice that wraps, bug or intention?
Hi! I just stumbled across something in Python 2.6 where I'm not sure if it is by design or a fault: x = 'abdc' x[-3:-3] - '' x[-3:-2] - 'b' x[-3:-1] - 'bc' x[-3: 0] - '' The one that actually bothers me here is the last one, I would have expected it to yield 'bcd' instead, because otherwise I don't see a way to specify a slice that starts with a negative index but still includes the last element. Similarly, I would expect x[-1,1] to yield 'ca' or at least raise an error, but not to return an empty string. Bug? Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ** Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ** Visit our website at http://www.satorlaser.de/ ** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. ** ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sequence slice that wraps, bug or intention?
2009/4/3 Ulrich Eckhardt eckha...@satorlaser.com: Hi! I just stumbled across something in Python 2.6 where I'm not sure if it is by design or a fault: x = 'abdc' x[-3:-3] - '' x[-3:-2] - 'b' x[-3:-1] - 'bc' x[-3: 0] - '' The one that actually bothers me here is the last one, I would have expected it to yield 'bcd' instead, because otherwise I don't see a way to specify a slice that starts with a negative index but still includes the last element. Similarly, I would expect x[-1,1] to yield 'ca' or at least raise an error, but not to return an empty string. Bug? Feature. Documented behaviour, even (http://docs.python.org/reference/expressions.html#id5 section Slicings). This question is more appropriate for python-list (comp.lang.python) as it is about using Python, rather than the development of the Python interpreter itself (although I can see that your uncertainty as to whether this was a bug might have led you to think this was a more appropriate list). You should first confirm on python-list that a given behaviour is a bug, and if it is, post it to the tracker, rather than to python-dev. In this case, the behaviour is fine. As regards your point I don't see a way to specify a slice that starts with a negative index but still includes the last element what you want is x[-3:]. If you want to discuss this further, please do so on python-list. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
2009/4/3 Hrvoje Niksic hrvoje.nik...@avl.com:
I've stumbled upon an oddity using sets. It's trivial to test if a value is
in the set, but it appears to be impossible to retrieve a stored value,
other than by iterating over the whole set. Let me describe a concrete use
case.
Imagine a set of objects identified by some piece of information, such as a
key slot (guaranteed to be constant for any particular element). The
object could look like this:
class Element(object):
def __init__(self, key):
self.key = key
def __eq__(self, other):
return self.key == other
def __hash__(self):
return hash(self.key)
# ...
Now imagine a set s of such objects. I can add them to the set:
s = set()
s.add(Element('foo'))
s.add(Element('bar'))
I can test membership using the keys:
'foo' in s
True
'blah' in s
False
But I can't seem to find a way to retrieve the element corresponding to
'foo', at least not without iterating over the entire set. Is this an
oversight or an intentional feature? Or am I just missing an obvious way to
do this?
My instinct is that it's intentional. I'd say that you're abusing
__eq__ here. If you can say x in s and then can't use x as if it
were the actual item inserted into s, then are they really equal?
Using a dict seems like the correct answer. I certainly don't think
it's worth complicating the set interface to cover this corner case.
Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Getting values stored inside sets
I've stumbled upon an oddity using sets. It's trivial to test if a
value is in the set, but it appears to be impossible to retrieve a
stored value, other than by iterating over the whole set. Let me
describe a concrete use case.
Imagine a set of objects identified by some piece of information, such
as a key slot (guaranteed to be constant for any particular element).
The object could look like this:
class Element(object):
def __init__(self, key):
self.key = key
def __eq__(self, other):
return self.key == other
def __hash__(self):
return hash(self.key)
# ...
Now imagine a set s of such objects. I can add them to the set:
s = set()
s.add(Element('foo'))
s.add(Element('bar'))
I can test membership using the keys:
'foo' in s
True
'blah' in s
False
But I can't seem to find a way to retrieve the element corresponding to
'foo', at least not without iterating over the entire set. Is this an
oversight or an intentional feature? Or am I just missing an obvious
way to do this?
I know I can work around this by changing the set of elements to a dict
that maps key - element, but this feels unsatisfactory. It's
redundant, as the element already contains all the necessary
information, and the set already knows how to use it, and the set must
remember the original elements anyway, to be able to iterate over them,
so why not allow one to retrieve them? Secondly, the data structure I
need conceptually *is* a set of elements, so it feels wrong to
pigeonhole it into a dict.
This wasn't an isolated case, we stumbled on this several times while
trying to use sets. In comparison, STL sets don't have this limitation.
If this is not possible, I would like to propose either that set's
__getitem__ translates key to value, so that s['foo'] would return the
first element, or, if this is considered ugly, an equivalent method,
such as s.get('foo').
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
Hrvoje Niksic wrote:
I've stumbled upon an oddity using sets. It's trivial to test if a
value is in the set, but it appears to be impossible to retrieve a
stored value,
Set elements, by definition, do not have keys or position by which to
grab. When they do, use a dict or list.
other than by iterating over the whole set. Let me
describe a concrete use case.
Imagine a set of objects identified by some piece of information, such
as a key slot (guaranteed to be constant for any particular element).
The object could look like this:
class Element(object):
def __init__(self, key):
self.key = key
def __eq__(self, other):
return self.key == other
def __hash__(self):
return hash(self.key)
# ...
Now imagine a set s of such objects. I can add them to the set:
s = set()
s.add(Element('foo'))
s.add(Element('bar'))
I can test membership using the keys:
'foo' in s
True
'blah' in s
False
But I can't seem to find a way to retrieve the element corresponding to
'foo', at least not without iterating over the entire set. Is this an
oversight or an intentional feature? Or am I just missing an obvious
way to do this?
Use a dict, like you did.
I know I can work around this by changing the set of elements to a dict
that maps key - element, but this feels unsatisfactory.
Sorry, that is the right way.
It's
redundant, as the element already contains all the necessary
information,
Records in a database have all the information of the record, but we
still put out fields for indexes.
tjr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package Management - thoughts from the peanut gallery
2009/4/3 Tarek Ziadé ziade.ta...@gmail.com: Guys, The tasks discussed so far are: - version definition (http://wiki.python.org/moin/DistutilsVersionFight) - egg.info standardification (PEP 376) - metadata enhancement (rewrite PEP 345) - static metadata definition work (*) Looks fine ... and very useful ... ;) - creation of a network of OS packager people - PyPI mirroring (PEP 381) Wow ! BTW ... I see nothing about removing dist_* commands from distutils ... Q: Am I wrong or it seems they will remain in stdlib ? -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
On Fri, 3 Apr 2009 11:22:02 pm Paul Moore wrote:
I'd say that you're abusing __eq__ here. If you can say x in s
and then can't use x as if it were the actual item inserted into
s, then are they really equal?
That's hardly unusual in Python.
alist = [0, 1, 2, 3, 4]
3.0 in alist
True
alist[3.0]
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: list indices must be integers
Besides, there's a concrete use-case for retrieving the actual object
inside the set. You can ensure that you only have one instance of any
object with a particular value by using a cache like this:
_cache = {}
def cache(obj):
if obj in _cache: return _cache[obj]
_cache[obj] = obj
return obj
Arguably, it would be neater if the cache was a set rather than a dict,
thus saving one pointer per item, but of course that would rely on a
change on set behaviour.
--
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
On Sat, 4 Apr 2009 02:07:28 am Antoine Pitrou wrote:
Your example is wrong:
Of course it is. The perils of posting at 2am, sorry.
Nevertheless, the principle still holds. There's nothing in Python that
prohibits two objects from being equal, but without them being
interchangeable. As poorly written as my example was, it still holds: I
just need to add a level of indirection.
alist = [100, 111, 102, 103, 105, 104, 106, 108]
indices_of_odd_numbers = [alist.index(n) for n in alist if n%2]
if Decimal('3') in indices_of_odd_numbers:
... print alist[Decimal('3')]
...
Traceback (most recent call last):
File stdin, line 2, in module
TypeError: list indices must be integers
Python does not promise that if x == y, you can use y anywhere you can
use x. Nor should it. Paul's declaration of abuse of __eq__ is
unfounded.
--
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On Fri, Apr 3, 2009 at 11:27, Antoine Pitrou solip...@pitrou.net wrote: Thomas Wouters thomas at python.org writes: Pystone is pretty much a useless benchmark. If it measures anything, it's the speed of the bytecode dispatcher (and it doesn't measure it particularly well.) PyBench isn't any better, in my experience. I don't think pybench is useless. It gives a lot of performance data about crucial internal operations of the interpreter. It is of course very little real-world, but conversely makes you know immediately where a performance regression has happened. (by contrast, if you witness a regression in a high-level benchmark, you still have a lot of investigation to do to find out where exactly something bad happened) Really? Have you tried it? I get at least 5% noise between runs without any changes. I have gotten results that include *negative* run times. And yes, I tried all the different settings for calibration runs and timing mechanisms. The tests in PyBench are not micro-benchmarks (they do way too much for that), they don't try to minimize overhead or noise, but they are also not representative of real-world code. That doesn't just mean you can't infer the affected operation from the test name, but you can't infer anything. You can just be looking at differently borrowed runtime. I have in the past written patches to Python that improved *every* micro-benchmark and *every* real-world measurement I made, except PyBench. Trying to pinpoint the slowdown invariably lead to tests that did too much in the measurement loop, introduced too much noise in the calibration run or just spent their time *in the measurement loop* on doing setup and teardown of the test. Collin and Jeffrey have seen the exact same thing since starting work on Unladen Swallow. So, sure, it might be useful if you have 10% or more difference across the board, and if you don't have access to anything but pybench and pystone. Perhaps someone should start maintaining a suite of benchmarks, high-level and low-level; we currently have them all scattered around (pybench, pystone, stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not to mention other third-party stuff that can be found in e.g. the Computer Language Shootout). That's exactly what Collin proposed at the summits last week. Have you seen http://code.google.com/p/unladen-swallow/wiki/Benchmarks ? Please feel free to suggest more benchmarks to add :) -- Thomas Wouters tho...@python.org Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] issue5578 - explanation
Guido van Rossum wrote: But anyways this is moot, the bug was only about exec in a class body *nested inside a function*. Indeed, I just hate seeing execs and it was an interesting mental exercise to try and get rid of the above one ;-) Assuming it breaks no tests, would there be objection to me committing the above change to the Python 3 trunk? That's up to Benjamin. Personally, I live by if it ain't broke, don't fix it. :-) Anything using an exec is broken by definition ;-) Benjamin? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package Management - thoughts from the peanut gallery
On Fri, Apr 3, 2009 at 8:36 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: On Fri, Apr 3, 2009 at 2:56 PM, Olemis Lang ole...@gmail.com wrote: BTW ... I see nothing about removing dist_* commands from distutils ... Q: Am I wrong or it seems they will remain in stdlib ? This is roughly what Guido was talking about when he said we would remove things like bdist_rpm from the stdlib : it's too OS-specific for the stdlib to do a good job in this area. To discuss this plan in details, let's move to Distutils-SIG understood ... ;) -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On Fri, Apr 3, 2009 at 2:27 AM, Antoine Pitrou solip...@pitrou.net wrote: Thomas Wouters thomas at python.org writes: Pystone is pretty much a useless benchmark. If it measures anything, it's the speed of the bytecode dispatcher (and it doesn't measure it particularly well.) PyBench isn't any better, in my experience. I don't think pybench is useless. It gives a lot of performance data about crucial internal operations of the interpreter. It is of course very little real-world, but conversely makes you know immediately where a performance regression has happened. (by contrast, if you witness a regression in a high-level benchmark, you still have a lot of investigation to do to find out where exactly something bad happened) Perhaps someone should start maintaining a suite of benchmarks, high-level and low-level; we currently have them all scattered around (pybench, pystone, stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not to mention other third-party stuff that can be found in e.g. the Computer Language Shootout). Already in the works :) As part of the common standard library and test suite that we agreed on at the PyCon language summit last week, we're going to include a common benchmark suite that all Python implementations can share. This is still some months off, though, so there'll be plenty of time to bikeshed^Wrationally discuss which benchmarks should go in there. Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package Management - thoughts from the peanut gallery
Tarek Ziadé wrote: I have taken the commitment to lead these tasks and synchronize the people that are willing to help on this. Good, I'm one of those people, sadly my only help may be to ask how is this bit going to be done?. The tasks discussed so far are: - version definition (http://wiki.python.org/moin/DistutilsVersionFight) - egg.info standardification (PEP 376) - metadata enhancement (rewrite PEP 345) - static metadata definition work (*) These all seem to be a subset of the last one, right? - creation of a network of OS packager people This would be useful... - PyPI mirroring (PEP 381) I don't see why PyPI isn't just ported to GAE with an S3 data storage bit and be done with it... Offline mirrors for people behind firewalls already have solutions out there... Each one of this task has a leader, except the one with (*). I just got back from travelling, and I will reorganize http://wiki.python.org/moin/Distutils asap to it is up-to-date. Cool, is this the focal point to track your activities? If you want to work on one of this task or feel there's a new task you can start, please, join Distutils SIG or contact me, Well, I think my big list breaks down roughly as tasks, of which I think the stuff you're already doing will hopefully take care of the first 2, but what about the rest. If labour shortage is all that's stopping this, then let me know ;-) Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
I think it's worse to give the poor guy the run around by making him run lots of random benchmarks. the poor guy works for Wingware (a company you may have heard of) and has contributed to Python at several occasions. His name is John Ehresmann. In the end, someone will run a timeit or have a specific case that shows the full effect. All of the respondents so far seem to have a clear intuition that hook is right in the middle of a critical path. Their intuition matches what I learned by spending a month trying to find ways to optimize dictionaries. Ok, so add me as a respondent who thinks that this deserves to be added despite being in the critical path. I doubt it will be noticeable in practice. Am surprised that there has been no discussion of why this should be in the default build (as opposed to a compile time option). Because, as a compile time option, it will be useless. It's not targeted for people who want to work on the Python VM (who are the primary users of compile time options), but for people developing Python applications. AFAICT, users have not previously requested a hook like this. That's because debugging Python in general is in a sad state (which, in turn, is because you can get very far with just print calls). Also, there has been no discussion for an overall strategy for monitoring containers in general. Lists and tuples will both defy this approach because there is so much code that accesses the arrays directly. Dicts are special because they are used to implement namespaces. Watchpoints is an incredibly useful debugging aid. Am not sure whether the setitem hook would work for other implementations either. I can't see why it shouldn't. If my thoughts on the subject bug you, I'll happily withdraw from the thread. I don't aspire to be a source of negativity. I just happen to think this proposal isn't a good idea. As somebody who has worked a lot on performance, I'm puzzled how easily you judge a the performance impact of a patch without having seen any benchmarks. If I have learned anything about performance, it is this: never guess the performance aspects of code without benchmarking. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
I just noticed that the new io-c modules were merged in the py3k branch (I know, I am kind late on the news—blame school work). Anyway, I am just wondering if it would be a good idea to put the io-c modules in a sub-directory (like sqlite), instead of scattering them around in the Modules/ directory. Welcome back! I have no particular opinion on this. I suggest waiting for Benjamin's advice and following it :-) I would suggest to leave it as is: a) never change a running system b) flat is better than nested Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Package Management - thoughts from the peanut gallery
On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers ch...@simplistix.co.uk wrote: Tarek Ziadé wrote: - PyPI mirroring (PEP 381) I don't see why PyPI isn't just ported to GAE with an S3 data storage bit and be done with it... Offline mirrors for people behind firewalls already have solutions out there... -1 ... IMHO ... -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: No me gustan los templates de Django ... -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
I've stumbled upon an oddity using sets. It's trivial to test if a
value is in the set, but it appears to be impossible to retrieve a
stored value, other than by iterating over the whole set.
Of course it is. That's why it is called a set: it's an unordered
collection of objects, keyed by nothing.
If you have a set of elements, and you check 'foo' in s, then
you should be able just to use the string 'foo' itself for whatever
you want to do with it - you have essentially created a set of
strings. If you think that 'foo' and Element('foo') are different
things, you should not implement __eq__ in a way that they are
considered equal.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
Thomas Wouters thomas at python.org writes: Really? Have you tried it? I get at least 5% noise between runs without any changes. I have gotten results that include *negative* run times. That's an implementation problem, not an issue with the tests themselves. Perhaps a better timing mechanism could be inspired from the timeit module. Perhaps the default numbers of iterations should be higher (many subtests run in less than 100ms on a modern CPU, which might be too low for accurate measurement). Perhaps the so-called calibration should just be disabled. etc. The tests in PyBench are not micro-benchmarks (they do way too much for that), Then I wonder what you call a micro-benchmark. Should it involve direct calls to low-level C API functions? but they are also not representative of real-world code. Representativity is not black or white. Is measuring Spitfire performance representative of the Genshi templating engine, or str.format-based templating? Regardless of the answer, it is still an interesting measurement. That doesn't just mean you can't infer the affected operation from the test name I'm not sure what you mean by that. If you introduce an optimization to make list comprehensions faster, it will certainly show up in the list comprehensions subtest, and probably in none of the other tests. Isn't it enough in terms of specificity? Of course, some optimizations are interpreter-wide, and then the breakdown into individual subtests is less relevant. I have in the past written patches to Python that improved *every* micro-benchmark and *every* real-world measurement I made, except PyBench. Well, I didn't claim that pybench measures /everything/. That's why we have other benchmarks as well (stringbench, iobench, whatever). It does test a bunch of very common operations which are important in daily use of Python. If some important operation is missing, it's possible to add a new test. Conversely, someone optimizing e.g. list comprehensions and trying to measure the impact using a set of so-called real-world benchmarks which don't involve any list comprehension in their critical path will not see any improvement in those real-world benchmarks. Does it mean that the optimization is useless? No, certainly not. The world is not black and white. That's exactly what Collin proposed at the summits last week. Have you seen http://code.google.com/p/unladen-swallow/wiki/Benchmarks Yes, I've seen. I haven't tried it, I hope it can be run without installing the whole unladen-swallow suite? These are the benchmarks I've had a tendency to use depending on the issue at hand: pybench, richards, stringbench, iobench, binary-trees (from the Computer Language Shootout). And various custom timeit runs :-) Cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
Just want to reply quickly because I'm traveling -- I appreciate the feedback from Raymond and others. Part of the reason I created an issue with a proof of concept patch is to get this kind of feedback. I also agree that this shouldn't go in if it slows things down noticeably. I will do some benchmarking and look at the dtrace patches next week to see if there is some sort of more systematic way of adding these types of hooks. John ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package Management - thoughts from the peanut gallery
Olemis Lang wrote: On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers ch...@simplistix.co.uk wrote: Tarek Ziadé wrote: - PyPI mirroring (PEP 381) I don't see why PyPI isn't just ported to GAE with an S3 data storage bit and be done with it... Offline mirrors for people behind firewalls already have solutions out there... -1 ... IMHO ... For what reason? Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
2009/4/3 Steven D'Aprano st...@pearwood.info:
Python does not promise that if x == y, you can use y anywhere you can
use x. Nor should it. Paul's declaration of abuse of __eq__ is
unfounded.
Sorry, I was trying to simplify what I was saying, and simplified it
to the point where it didn't make sense :-) Martin (quoted below)
explained what I was trying to say far more clearly.
2009/4/3 Martin v. Löwis mar...@v.loewis.de:
If you have a set of elements, and you check 'foo' in s, then
you should be able just to use the string 'foo' itself for whatever
you want to do with it - you have essentially created a set of
strings. If you think that 'foo' and Element('foo') are different
things, you should not implement __eq__ in a way that they are
considered equal.
-- in particular, if you're using things in sets (which are *all
about* equality, insofar as that's how duplicates are defined) you
should ensure that your definition of __eq__ respects the idea that
equal objects are duplicates (ie, interchangeable). Otherwise, a dict
is the appropriate data structure.
Actually, given the definition in the original post,
class Element(object):
def __init__(self, key):
self.key = key
def __eq__(self, other):
return self.key == other
def __hash__(self):
return hash(self.key)
as far as I can tell, equality is *only* defined between Elements and
keys - not even between 2 elements! So with that definition, there
could be many Elements in a set, all equal to the same key. Which is
completely insane.
In fact, Python seems to be doing something I don't understand:
class Element(object):
...def __init__(self, key, id):
...self.key = key
...self.id = id
...def __eq__(self, other):
...print Calling __eq__ for %s % self.id
...return self.key == other
...def __hash__(self):
...return hash(self.key)
...
a = Element('k', 'a')
b = Element('k', 'b')
a == b
Calling __eq__ for a
Calling __eq__ for b
True
a == a
Calling __eq__ for a
Calling __eq__ for a
True
Why does __eq__ get called twice in these cases? Why does a == b, as
that means a.key == b, and clearly a.key ('k') does *not* equal b. Or
are there some further options being tried, in str,__eq__ or
object.__eq__? The documentation doesn't say so... Specifically,
there's nothing saying that a reversed version is tried.
Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On 2009-04-03 18:06, Thomas Wouters wrote: On Fri, Apr 3, 2009 at 11:27, Antoine Pitrou solip...@pitrou.net wrote: Thomas Wouters thomas at python.org writes: Pystone is pretty much a useless benchmark. If it measures anything, it's the speed of the bytecode dispatcher (and it doesn't measure it particularly well.) PyBench isn't any better, in my experience. I don't think pybench is useless. It gives a lot of performance data about crucial internal operations of the interpreter. It is of course very little real-world, but conversely makes you know immediately where a performance regression has happened. (by contrast, if you witness a regression in a high-level benchmark, you still have a lot of investigation to do to find out where exactly something bad happened) Really? Have you tried it? I get at least 5% noise between runs without any changes. I have gotten results that include *negative* run times. On which platform ? pybench 2.0 works reasonably well on Linux and Windows, but of course can't do better than the timers available for those platforms. If you have e.g. NTP running and it uses wall clock timers, it is possible that you get negative round times. If you don't and still get negative round times, you have to change the test parameters (see below). And yes, I tried all the different settings for calibration runs and timing mechanisms. The tests in PyBench are not micro-benchmarks (they do way too much for that), they don't try to minimize overhead or noise, That is not true. They were written as micro-benchmarks and adjusted to have a high signal-noise ratio. For some operations this isn't easy to do, but I certainly tried hard to get the overhead low (note that the overhead is listed in the output). That said, please keep in mind that the settings in pybench were last adjusted some years ago to have the tests all run in more or less the same wall clock time. CPUs have evolved a lot since then and this shows. but they are also not representative of real-world code. True and they never were meant for that, since I was frustrated by other benchmarks at the time and the whole approach in general. Each of the tests checks one specific aspect of Python. If your application happens to use a lot of dictionary operations, you'll be mostly interested in those. If you do a lot of simple arithmetic, there's another test for that. On top of that the application is written to be easily extensible, so it's easy to add new tests specific to whatever application space you're after. That doesn't just mean you can't infer the affected operation from the test name, but you can't infer anything. You can just be looking at differently borrowed runtime. I have in the past written patches to Python that improved *every* micro-benchmark and *every* real-world measurement I made, except PyBench. Trying to pinpoint the slowdown invariably lead to tests that did too much in the measurement loop, introduced too much noise in the calibration run or just spent their time *in the measurement loop* on doing setup and teardown of the test. pybench calibrates itself to remove that kind of noise from the output. Each test has a .calibrate() method which does all the setup and tear down minus the actual benchmark operations. If you get wrong numbers, try adjusting the parameters and add more packets of operations. Don't forget to adjust the version number to not compare apples and orange, though. Perhaps it's time to readjust the pybench parameters to todays CPUs. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 03 2009) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package Management - thoughts from the peanut gallery
On Fri, Apr 3, 2009 at 6:20 PM, Chris Withers ch...@simplistix.co.uk wrote: Tarek Ziadé wrote: I have taken the commitment to lead these tasks and synchronize the people that are willing to help on this. Good, I'm one of those people, Great ! sadly my only help may be to ask how is this bit going to be done?. I'll work on the wiki this week end for that The tasks discussed so far are: - version definition (http://wiki.python.org/moin/DistutilsVersionFight) - egg.info standardification (PEP 376) - metadata enhancement (rewrite PEP 345) - static metadata definition work (*) These all seem to be a subset of the last one, right? Sorry I used task I should have used topics. We are trying to have a list of well-defined, isolated tasks. Theses tasks are built upon the discussions we have in these topics. The last topic (static metadata) might generate new tasks and/or complete existing tasks. - PyPI mirroring (PEP 381) I don't see why PyPI isn't just ported to GAE with an S3 data storage bit and be done with it... Offline mirrors for people behind firewalls already have solutions out there... GAE+S3 is just an implementation imho. We still need a mirroring protocol ala CPAN and features in client softwares to use them. (as defined in 381) Each one of this task has a leader, except the one with (*). I just got back from travelling, and I will reorganize http://wiki.python.org/moin/Distutils asap to it is up-to-date. Cool, is this the focal point to track your activities? Exactly. And Distutils-SIG is the mailing list to discuss in ;) If you want to work on one of this task or feel there's a new task you can start, please, join Distutils SIG or contact me, Well, I think my big list breaks down roughly as tasks, of which I think the stuff you're already doing will hopefully take care of the first 2, but what about the rest. If labour shortage is all that's stopping this, then let me know ;-) Please discuss these new points in Distutils-SIG Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package Management - thoughts from the peanut gallery
Chris Withers wrote: Olemis Lang wrote: On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers ch...@simplistix.co.uk wrote: Tarek Ziadé wrote: - PyPI mirroring (PEP 381) I don't see why PyPI isn't just ported to GAE with an S3 data storage bit and be done with it... Offline mirrors for people behind firewalls already have solutions out there... -1 ... IMHO ... For what reason? GAE does suffer from blackouts - which is the problem we are attempting to solve with mirroring. I don't see why we should tie vital Python infrastructure to the proprietary APIs of a single vendor and outsource delivery entirely to them. If we have the manpower to do this ourselves it seems better to do it and retain control. Added to which GAE is a commercial service and beyond a certain level bandwidth / cycles needs paying for. This may not be an issue in itself (either Google may waive charges or the PSF may be willing to pay). Michael Chris -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On Fri, Apr 3, 2009 at 9:43 AM, Antoine Pitrou solip...@pitrou.net wrote: Thomas Wouters thomas at python.org writes: Really? Have you tried it? I get at least 5% noise between runs without any changes. I have gotten results that include *negative* run times. That's an implementation problem, not an issue with the tests themselves. Perhaps a better timing mechanism could be inspired from the timeit module. Perhaps the default numbers of iterations should be higher (many subtests run in less than 100ms on a modern CPU, which might be too low for accurate measurement). Perhaps the so-called calibration should just be disabled. etc. The tests in PyBench are not micro-benchmarks (they do way too much for that), Then I wonder what you call a micro-benchmark. Should it involve direct calls to low-level C API functions? I agree that a suite of microbenchmarks is supremely useful: I would very much like to be able to isolate, say, raise statement performance. PyBench suffers from implementation defects that in its current incarnation make it unsuitable for this, though: - It does not effectively isolate component performance as it claims. When I was working on a change to BINARY_MODULO to make string formatting faster, PyBench would report that floating point math got slower, or that generator yields got slower. There is a lot of random noise in the results. - We have observed overall performance swings of 10-15% between runs on the same machine, using the same Python binary. Using the same binary on the same unloaded machine should give as close an answer to 0% as possible. - I wish PyBench actually did more isolation. Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it didn't put keyword arguments and **kwargs in the same microbenchmark. - In experimenting with gcc 4.4's FDO support, I produced a training load that resulted in a 15-30% performance improvement (depending on benchmark) across all benchmarks. Using this trained binary, PyBench slowed down by 10%. - I would like to see PyBench incorporate better statistics for indicating the significance of the observed performance difference. I don't believe that these are insurmountable problems, though. A great contribution to Python performance work would be an improved version of PyBench that corrects these problems and offers more precise measurements. Is that something you might be interested in contributing to? As performance moves more into the wider consciousness, having good tools will become increasingly important. Thanks, Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
Collin Winter wrote: On Fri, Apr 3, 2009 at 2:27 AM, Antoine Pitrou solip...@pitrou.net wrote: Thomas Wouters thomas at python.org writes: Pystone is pretty much a useless benchmark. If it measures anything, it's the speed of the bytecode dispatcher (and it doesn't measure it particularly well.) PyBench isn't any better, in my experience. I don't think pybench is useless. It gives a lot of performance data about crucial internal operations of the interpreter. It is of course very little real-world, but conversely makes you know immediately where a performance regression has happened. (by contrast, if you witness a regression in a high-level benchmark, you still have a lot of investigation to do to find out where exactly something bad happened) Perhaps someone should start maintaining a suite of benchmarks, high-level and low-level; we currently have them all scattered around (pybench, pystone, stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not to mention other third-party stuff that can be found in e.g. the Computer Language Shootout). Already in the works :) As part of the common standard library and test suite that we agreed on at the PyCon language summit last week, we're going to include a common benchmark suite that all Python implementations can share. This is still some months off, though, so there'll be plenty of time to bikeshed^Wrationally discuss which benchmarks should go in there. Where is the right place for us to discuss this common benchmark and test suite? As the benchmark is developed I would like to ensure it can run on IronPython. The test suite changes will need some discussion as well - Jython and IronPython (and probably PyPy) have almost identical changes to tests that currently rely on deterministic finalisation (reference counting) so it makes sense to test changes on both platforms and commit a single solution. Michael Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
On Fri, 3 Apr 2009 at 17:57, Paul Moore wrote:
In fact, Python seems to be doing something I don't understand:
class Element(object):
...def __init__(self, key, id):
...self.key = key
...self.id = id
...def __eq__(self, other):
...print Calling __eq__ for %s % self.id
...return self.key == other
...def __hash__(self):
...return hash(self.key)
...
a = Element('k', 'a')
b = Element('k', 'b')
a == b
Calling __eq__ for a
Calling __eq__ for b
True
a == a
Calling __eq__ for a
Calling __eq__ for a
True
Why does __eq__ get called twice in these cases? Why does a == b, as
that means a.key == b, and clearly a.key ('k') does *not* equal b. Or
are there some further options being tried, in str,__eq__ or
object.__eq__? The documentation doesn't say so... Specifically,
there's nothing saying that a reversed version is tried.
a == b
So, python calls a.__eq__(b)
Now, that function does:
a.key == b
Since b is an object with an __eq__ method, python calls
b.__eq__(a.key).
That function does:
a.key == b.key
ie: the OP's code is inefficient :)
--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: Collin Winter wrote: As part of the common standard library and test suite that we agreed on at the PyCon language summit last week, we're going to include a common benchmark suite that all Python implementations can share. This is still some months off, though, so there'll be plenty of time to bikeshed^Wrationally discuss which benchmarks should go in there. Where is the right place for us to discuss this common benchmark and test suite? As the benchmark is developed I would like to ensure it can run on IronPython. The test suite changes will need some discussion as well - Jython and IronPython (and probably PyPy) have almost identical changes to tests that currently rely on deterministic finalisation (reference counting) so it makes sense to test changes on both platforms and commit a single solution. I believe Brett Cannon is the best person to talk to about this kind of thing. I don't know that any common mailing list has been set up, though there may be and Brett just hasn't told anyone yet :) Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
Collin Winter collinw at gmail.com writes: - I wish PyBench actually did more isolation. Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it didn't put keyword arguments and **kwargs in the same microbenchmark. Well, there is a balance to be found between having more subtests and keeping a reasonable total running time :-) (I have to plead guilty for ComplexPythonFunctionCalls, btw) - I would like to see PyBench incorporate better statistics for indicating the significance of the observed performance difference. I see you already have this kind of measurement in your perf.py script, would it be easy to port it? We could also discuss making individual tests longer (by changing the default warp factor). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
2009/4/3 R. David Murray rdmur...@bitdance.com: a == b So, python calls a.__eq__(b) Now, that function does: a.key == b Since b is an object with an __eq__ method, python calls b.__eq__(a.key). That's the bit I can't actually find documented anywhere. Ah, looking again I see that I misread the section describing the rich comparison methods: There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, __lt__() and __gt__() are each other’s reflection, __le__() and __ge__() are each other’s reflection, and __eq__() and __ne__() are their own reflection. I read that as meaning that no reversed version was called, whereas it actually means that __eq__ is its own reversed version - and so gets called both times. Thanks for helping me clear that up! Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On Fri, Apr 3, 2009 at 10:50 AM, Antoine Pitrou solip...@pitrou.net wrote: Collin Winter collinw at gmail.com writes: - I wish PyBench actually did more isolation. Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it didn't put keyword arguments and **kwargs in the same microbenchmark. Well, there is a balance to be found between having more subtests and keeping a reasonable total running time :-) (I have to plead guilty for ComplexPythonFunctionCalls, btw) Sure, there's definitely a balance to maintain. With perf.py, we're going down the road of having different tiers of benchmarks: the default set is the one we pay the most attention to, with other benchmarks available for benchmarking certain specific subsystems or workloads (like pickling list-heavy input data). Something similar could be done for PyBench, giving the user the option of increasing the level of detail (and run-time) as appropriate. - I would like to see PyBench incorporate better statistics for indicating the significance of the observed performance difference. I see you already have this kind of measurement in your perf.py script, would it be easy to port it? Yes, it should be straightforward to incorporate these statistics into PyBench. In the same directory as perf.py, you'll find test_perf.py which includes tests for the stats functions we're using. Collin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
Perhaps we could add something like a sys.namespace_packages that would be updated by this mechanism? Then, pkg_resources could check both that and its internal registry to be both backward and forward compatible. I could see no problem with that, so I have added this to the PEP. Thanks for the feedback, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
Chris Withers wrote: Martin v. Löwis wrote: I propose the following PEP for inclusion to Python 3.1. Please comment. Would this support the following case: I have a package called mortar, which defines useful stuff: from mortar import content, ... I now want to distribute large optional chunks separately, but ideally so that the following will will work: from mortar.rbd import ... from mortar.zodb import ... from mortar.wsgi import ... Does the PEP support this? That's the primary purpose of the PEP. You can do this today already (see the zope package, and the reference to current techniques in the PEP), but the PEP provides a cleaner way. In each chunk (which the PEP calls portion), you had a structure like this: mortar/ mortar/rbd.pkg (contains just *) mortar/rbd.py or mortar/ mortar/zobd.pkg mortar/zobd/ mortar/zobd/__init__.py mortar/zobd/backends.py As a site effect, you can also do import mortar, but that would just give you the (nearly) empty namespace package, whose only significant contents is the variable __path__. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyDict_SetItem hook
On Thu, Apr 2, 2009 at 5:57 PM, Guido van Rossum gu...@python.org wrote: On Thu, Apr 2, 2009 at 3:07 PM, Raymond Hettinger pyt...@rcn.com wrote: Wow. Can you possibly be more negative? I think it's worse to give the poor guy the run around Mind your words please. by making him run lots of random benchmarks. In the end, someone will run a timeit or have a specific case that shows the full effect. All of the respondents so far seem to have a clear intuition that hook is right in the middle of a critical path. Their intuition matches what I learned by spending a month trying to find ways to optimize dictionaries. Am surprised that there has been no discussion of why this should be in the default build (as opposed to a compile time option). AFAICT, users have not previously requested a hook like this. I may be partially to blame for this. John and Stephan are requesting this because it would (mostly) fulfill one of the top wishes of the users of Wingware. So the use case is certainly real. Also, there has been no discussion for an overall strategy for monitoring containers in general. Lists and tuples will both defy this approach because there is so much code that accesses the arrays directly. Am not sure whether the setitem hook would work for other implementations either. The primary use case is some kind of trap on assignment. While this cannot cover all cases, most non-local variables are stored in dicts. List mutations are not in the same league, as use case. It seems weird to me that Collin's group can be working so hard just to get a percent or two improvement in specific cases for pickling while python-dev is readily entertaining a patch that slows down the entire language. I don't actually believe that you can know whether this affects performance at all without serious benchmarking. The patch amounts to a single global flag check as long as the feature is disabled, and that flag could be read from the L1 cache. When I was optimizing the tracing support in the eval loop, we started with two memory loads and an if test. Removing the whole thing saved about 3% of runtime, although I think that had been as high as 5% when Neal measured it a year before. (That indicates that the exact arrangement of the code can affect performance in subtle and annoying ways.) Removing one of the two loads saved about 2% of runtime. I don't remember exactly which benchmark that was; it may just have been pybench. Here, we're talking about introducing a load+if in dicts, which is less critical than the eval loop, so I'd guess that the effect will be less than 2% overall. I do think the real-life benchmarks are worth getting for this, but they may not predict the effect after other code changes. And I don't really have an opinion on what performance hit for normal use is worth better debugging. If my thoughts on the subject bug you, I'll happily withdraw from the thread. I don't aspire to be a source of negativity. I just happen to think this proposal isn't a good idea. I think we need more proof either way. Raymond - Original Message - From: Guido van Rossum gu...@python.org To: Raymond Hettinger pyt...@rcn.com Cc: Thomas Wouters tho...@python.org; John Ehresman j...@wingware.com; python-dev@python.org Sent: Thursday, April 02, 2009 2:19 PM Subject: Re: [Python-Dev] PyDict_SetItem hook Wow. Can you possibly be more negative? 2009/4/2 Raymond Hettinger pyt...@rcn.com: The measurements are just a distractor. We all already know that the hook is being added to a critical path. Everyone will pay a cost for a feature that few people will use. This is a really bad idea. It is not part of a thorough, thought-out framework of container hooks (something that would need a PEP at the very least). The case for how it helps us is somewhat thin. The case for DTrace hooks was much stronger. If something does go in, it should be #ifdef'd out by default. But then, I don't think it should go in at all. Raymond On Thu, Apr 2, 2009 at 04:16, John Ehresman j...@wingware.com wrote: Collin Winter wrote: Have you measured the impact on performance? I've tried to test using pystone, but am seeing more differences between runs than there is between python w/ the patch and w/o when there is no hook installed. The highest pystone is actually from the binary w/ the patch, which I don't really believe unless it's some low level code generation affect. The cost is one test of a global variable and then a switch to the branch that doesn't call the hooks. I'd be happy to try to come up with better numbers next week after I get home from pycon. Pystone is pretty much a useless benchmark. If it measures anything, it's the speed of the bytecode dispatcher (and it doesn't measure it particularly well.) PyBench isn't any better, in my experience. Collin has collected a set of reasonable benchmarks for Unladen Swallow, but they still
Re: [Python-Dev] PEP 382: Namespace Packages
On 08:15 pm, mar...@v.loewis.de wrote: Note that there is no such thing as a defining namespace package -- namespace package contents are symmetrical peers. With the PEP, a defining package becomes possible - at most one portion can define an __init__.py. For what it's worth, this is a _super_ useful feature for Twisted. We have one defining package for the twisted package (twisted core) and then a bunch of other things which want to put things into twisted.* (twisted.web, twisted.conch, et. al.). For debian we already have separate packages, but such a definition of namespace packages would allow us to actually have things separated out on the cheeseshop as well. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
2009/4/3 Antoine Pitrou solip...@pitrou.net: Alexandre Vassalotti alexandre at peadrop.com writes: I just noticed that the new io-c modules were merged in the py3k branch (I know, I am kind late on the news—blame school work). Anyway, I am just wondering if it would be a good idea to put the io-c modules in a sub-directory (like sqlite), instead of scattering them around in the Modules/ directory. Welcome back! I have no particular opinion on this. I suggest waiting for Benjamin's advice and following it :-) I'm +.2. This is the layout I would suggest: Modules/ _io/ _io.c stringio.c textio.c etc (unless the FLUFL wants to chime in) Benjamin-makes-boring-decisions-easy'ly yrs, Antoine. mad-with-power'ly yours, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
2009/4/3 Martin v. Löwis mar...@v.loewis.de: I just noticed that the new io-c modules were merged in the py3k branch (I know, I am kind late on the news—blame school work). Anyway, I am just wondering if it would be a good idea to put the io-c modules in a sub-directory (like sqlite), instead of scattering them around in the Modules/ directory. Welcome back! I have no particular opinion on this. I suggest waiting for Benjamin's advice and following it :-) I would suggest to leave it as is: a) never change a running system b) flat is better than nested It doesn't make sense, though, to have the 8 files that make up the _io module scattered around in a directory with scores of other ones. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
At 10:15 PM 4/3/2009 +0200, Martin v. Löwis wrote:
I should make it clear that this is not the case. I envision it to work
this way: import zope
- searches sys.path, until finding either a directory zope, or a file
zope.{py,pyc,pyd,...}
- if it is a directory, it checks for .pkg files. If it finds any,
it processes them, extending __path__.
- it *then* checks for __init__.py, taking the first hit anywhere
on __path__ (just like any module import would)
- if no .pkg was found, nor an __init__.py, it proceeds with the next
sys.path item (skipping the directory entirely)
Ah, I missed that. Maybe the above should be added to the PEP to clarify.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] issue5578 - explanation
2009/4/3 Chris Withers ch...@simplistix.co.uk: Guido van Rossum wrote: But anyways this is moot, the bug was only about exec in a class body *nested inside a function*. Indeed, I just hate seeing execs and it was an interesting mental exercise to try and get rid of the above one ;-) Assuming it breaks no tests, would there be objection to me committing the above change to the Python 3 trunk? That's up to Benjamin. Personally, I live by if it ain't broke, don't fix it. :-) Anything using an exec is broken by definition ;-) practicality beats purity Benjamin? +0 -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
On Fri, Apr 3, 2009 at 2:15 PM, Benjamin Peterson benja...@python.org wrote: 2009/4/3 Martin v. Löwis mar...@v.loewis.de: I just noticed that the new io-c modules were merged in the py3k branch (I know, I am kind late on the news—blame school work). Anyway, I am just wondering if it would be a good idea to put the io-c modules in a sub-directory (like sqlite), instead of scattering them around in the Modules/ directory. Welcome back! I have no particular opinion on this. I suggest waiting for Benjamin's advice and following it :-) I would suggest to leave it as is: a) never change a running system b) flat is better than nested It doesn't make sense, though, to have the 8 files that make up the _io module scattered around in a directory with scores of other ones. I think Benjamin is right. While most of the C source is indeed exactly one level below the root, there's plenty of code that isn't, e.g. _ctypes, cjkcodecs, expat, _multiprocessing, zlib. And even Objects/stringlib. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Integrate BeautifulSoup into stdlib?
Op dinsdag 24-03-2009 om 20:51 uur [tijdzone +0100], schreef Martin v. Löwis: The Windows story is indeed sad, as none of the Windows packaging formats provides support for dependencies That's not entirely true; Cygwin comes with a package management tool that probably could be used to set up a repository of python packages for native Windows: http://sources.redhat.com/cygwin-apps/setup.html This package manager is in no way dependent on Cygwin, supports (basic) dependencies, etc. Of course some people would have to take care of the packaging work (just like happens for most open source OS distros and for Mac OS X already). It seems like XEmacs is already using a fork of that installer for the same purpose. -- Jan Claeys ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] issue5578 - explanation
On Tue, Mar 31, 2009 at 11:25 PM, Guido van Rossum gu...@python.org wrote: Well hold on for a minute, I remember we used to have an exec statement in a class body in the standard library, to define some file methods in socket.py IIRC. FYI, collections.namedtuple is also implemented using exec. - Alexandre ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
On Fri, Apr 3, 2009 at 8:07 AM, Hrvoje Niksic hrvoje.nik...@avl.com wrote: But I can't seem to find a way to retrieve the element corresponding to 'foo', at least not without iterating over the entire set. Is this an oversight or an intentional feature? Or am I just missing an obvious way to do this? query_obj in s True s_prime = s.copy() s_prime.discard(query_obj) x = s.difference(s_prime).pop() Pretty ugly, but I think it only uses a shallow copy, and it might be a bit better than iterating, if difference is intelligent. I haven't run any tests though. -- Cheers, Leif ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
On Fri, Apr 3, 2009 at 13:15, Martin v. Löwis mar...@v.loewis.de wrote:
Note that there is no such thing as a defining namespace package --
namespace package contents are symmetrical peers.
With the PEP, a defining package becomes possible - at most one
portion can define an __init__.py.
I know that the current mechanisms don't support it, and it might
not be useful in general, but now there is a clean way of doing it,
so I wouldn't exclude it. Distribution-wise, all distributions
relying on the defining package would need to require (or
install_require, or depend on) it.
The above are also true for using only a '*' in .pkg files -- in that
event there are no sys.path changes. (Frankly, I'm doubtful that
anybody is using extend_path and .pkg files to begin with, so I'd be
fine with a proposal that instead used something like '.nsp' files that
didn't even need to be opened and read -- which would let the directory
scan stop at the first .nsp file found.
That would work for me as well. Nobody at PyCon could remember where
.pkg files came from.
I believe the PEP does this as well, IIUC.
Correct.
* It's possible to have a defining package dir and add-one package
dirs.
Also possible in the PEP, although the __init__.py must be in the first
such directory on sys.path.
I should make it clear that this is not the case. I envision it to work
this way: import zope
- searches sys.path, until finding either a directory zope, or a file
zope.{py,pyc,pyd,...}
- if it is a directory, it checks for .pkg files. If it finds any,
it processes them, extending __path__.
- it *then* checks for __init__.py, taking the first hit anywhere
on __path__ (just like any module import would)
Just so people know how this __init__ search could be done such that
__path__ is set from the .pkg is to treat it as a reload (assuming .pkg
files can only be found off of sys.path).
-Brett
- if no .pkg was found, nor an __init__.py, it proceeds with the next
sys.path item (skipping the directory entirely)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] And the winner is...
Op maandag 30-03-2009 om 21:54 uur [tijdzone -0500], schreef Guido van Rossum: But is his humility enough to cancel out Linus's attitude? I hope not, or the /.-crowd would become desperate... ;-) -- Jan Claeys ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
On Fri, Apr 3, 2009 at 5:12 PM, Benjamin Peterson benja...@python.org wrote: I'm +.2. This is the layout I would suggest: Modules/ _io/ _io.c stringio.c textio.c etc That seems good to me. I opened an issue on the tracker and included a patch. http://bugs.python.org/issue5682 -- Alexandre ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Going offline for three months
In order to hunker down and get my thesis proposal done by its due date, I am disabling mail delivery for myself for all mail.python.org mailing lists for three months (sans python-committers so I don't accidentally commit when I shouldn't). If something comes up I should know about you can always email or IM me directly. See you all on July 1. Here is to hoping I don't suffer any withdrawal. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
Paul Moore wrote: 2009/4/3 R. David Murray rdmur...@bitdance.com: a == b So, python calls a.__eq__(b) Now, that function does: a.key == b Since b is an object with an __eq__ method, python calls b.__eq__(a.key). That's the bit I can't actually find documented anywhere. It doesn't quite work the way RDM desribed it - he missed a step. a == b So, python calls a.__eq__(b) Now, that function does: a.key == b which first calls a.key.__eq__(b) # This step was missing Since str has no idea what an Element is, that returns NotImplemented. Since __eq__ is defined as being commutative, the interpreter then tries b.__eq__(a.key). That function does: b.key == a.key which calls b.key.__eq__(a.key) which is a well defined string comparison and returns the expected answer. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] issue5578 - explanation
Alexandre Vassalotti wrote: On Tue, Mar 31, 2009 at 11:25 PM, Guido van Rossum gu...@python.org wrote: Well hold on for a minute, I remember we used to have an exec statement in a class body in the standard library, to define some file methods in socket.py IIRC. FYI, collections.namedtuple is also implemented using exec. Ah, but it uses exec ... in That is much safer than an unqualified exec (where the issue is what namespace it executes in, and, consequentially, what early binding is possible). The patch bans only unqualified exec, IIUC. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Integrate BeautifulSoup into stdlib?
That's not entirely true; Cygwin comes with a package management tool that probably could be used to set up a repository of python packages for native Windows: http://sources.redhat.com/cygwin-apps/setup.html Ah, ok. It has the big disadvantage of not being Microsoft-endorsed, though. In that sense, it feels very much like easy_install (which also does dependencies). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Going offline for three months
Brett Cannon br...@python.org writes: See you all on July 1. Here is to hoping I don't suffer any withdrawal. Ouch. Best of luck to you! -- \ “Giving every man a vote has no more made men wise and free | `\ than Christianity has made them good.” —Henry L. Mencken | _o__) | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting values stored inside sets
[Nick Coghlan] It doesn't quite work the way RDM desribed it - he missed a step. Thanks for the clarification. We ought to write-out the process somewhere in a FAQ. It may also be instructive to step through the recipe that answers the OP's original request, http://code.activestate.com/recipes/499299/ The call get_equivalent(set([1, 2, 3]), 2.0) wraps the 2.0 in a new object t and calls t in set([1,2,3]). The set.__contains__ method hashes t using t.__hash__(self) and checks for an exact match using t.__eq__(other). Both calls delegate to float objects but the latter also records the other that resulted in a successful equality test (i.e. 2 is the member of the set that matched the 2.0). The get_equivalent call then returns the matching value, 2.0. As far as I can tell, the technique is completely generic and lets you reach inside any function or container to retrieve the other value that is equivalent to self. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Going offline for three months
Brett Cannon wrote: In order to hunker down and get my thesis proposal done by its due date, I am disabling mail delivery for myself for all mail.python.org http://mail.python.org mailing lists for three months (sans python-committers so I don't accidentally commit when I shouldn't). If something comes up I should know about you can always email or IM me directly. See you all on July 1. Here is to hoping I don't suffer any withdrawal. We'll miss you. Hope you don't end up preferring Java. ;-) Michael -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] core python tests (was: Re: PyDict_SetItem hook)
On Fri, Apr 03, 2009 at 07:00:43PM +0100, Michael Foord wrote: - Collin Winter wrote: - On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord - fuzzy...@voidspace.org.uk wrote: - - Collin Winter wrote: - - As part of the common standard library and test suite that we agreed - on at the PyCon language summit last week, we're going to include a - common benchmark suite that all Python implementations can share. This - is still some months off, though, so there'll be plenty of time to - bikeshed^Wrationally discuss which benchmarks should go in there. - - - Where is the right place for us to discuss this common benchmark and test - suite? - - As the benchmark is developed I would like to ensure it can run on - IronPython. - - The test suite changes will need some discussion as well - Jython and - IronPython (and probably PyPy) have almost identical changes to tests that - currently rely on deterministic finalisation (reference counting) so it - makes sense to test changes on both platforms and commit a single - solution. - - - I believe Brett Cannon is the best person to talk to about this kind - of thing. I don't know that any common mailing list has been set up, - though there may be and Brett just hasn't told anyone yet :) - - Collin - - Which begs the question of whether we *should* have a separate mailing list. - - I don't think we discussed this specific point in the language summit - - although it makes sense. Should we have a list specifically for the test - / benchmarking or would a more general implementations-sig be appropriate? - - And is it really Brett who sets up mailing lists? My understanding is - that he is pulling out of stuff for a while anyway, so that he can do - Java / Phd type things... ;-) 'tis a sad loss for both Python-dev and the academic community... I vote for a separate mailing list -- 'python-tests'? -- but I don't know exactly how splintered to make the conversation. It probably belongs at python.org but if you want me to host it, I can. N.B. There are a bunch of GSoC projects to work on or with the CPython test framework (increase test coverage, write plugins to make it runnable in nose or py.test, etc.). I don't know that the students should be active participants in such a list, but the mentors should at least try to stay in the loop so that we don't completely waste our time. cheers, --titus -- C. Titus Brown, c...@msu.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Going offline for three months
On Fri, Apr 3, 2009 at 19:51, Michael Foord fuzzy...@voidspace.org.ukwrote: Brett Cannon wrote: In order to hunker down and get my thesis proposal done by its due date, I am disabling mail delivery for myself for all mail.python.org http://mail.python.org mailing lists for three months (sans python-committers so I don't accidentally commit when I shouldn't). If something comes up I should know about you can always email or IM me directly. See you all on July 1. Here is to hoping I don't suffer any withdrawal. We'll miss you. Hope you don't end up preferring Java. ;-) No, it would be more like JavaScript, but I don't see that happening either. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should the io-c modules be put in their own directory?
I think Benjamin is right. While most of the C source is indeed exactly one level below the root, there's plenty of code that isn't, e.g. _ctypes, cjkcodecs, expat, _multiprocessing, zlib. And even Objects/stringlib. It's fine with me either way. Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] core python tests
C. Titus Brown wrote: I vote for a separate mailing list -- 'python-tests'? -- but I don't know exactly how splintered to make the conversation. It probably belongs at python.org but if you want me to host it, I can. If too many things get moved off to SIGs there won't be anything left for python-dev to talk about ;) (Although in this case it makes sense, as I expect there will be developers involved in alternate implementations that would like to be part of the test suite discussion without having to sign up for the rest of python-dev) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Documenting the Py3k coercion rules (was Re: Getting values stored inside sets)
Raymond Hettinger wrote: [Nick Coghlan] It doesn't quite work the way RDM desribed it - he missed a step. Thanks for the clarification. We ought to write-out the process somewhere in a FAQ. The closest we currently have to that is the write-up of the coercion rules in 2.x: http://docs.python.org/reference/datamodel.html#id5 Unfortunately, that mixes in a lot of CPython specific special cases along with the old coerce() builtin that obscure the basic behaviour for __op__ and __rop__ pairs. Here's an initial stab at a write-up of the coercion rules for Py3k that is accurate without getting too CPython specific: Given a OP b, the coercion sequence is: 1. Try a.__op__(b) 2. If a.__op__ doesn't exist or the call returns NotImplemented, try b.__rop__(a) 3. If b.__rop__ doesn't exist or the call returns NotImplemented, raise TypeError identifying type(a) and type(b) as unsupported operands for OP 4. If step 1 or 2 is successful, then the result of the call is the value of the expression Given a OP= b the coercion sequence is: 1. Try a = a.__iop__(b) 2. If a.__iop__ doesn't exist or the call returns not implemented, try a = a OP b using the normal binary coercion rules above Special cases: - if type(b) is a strict subclass of type(a), then b.__rop__ is tried before a.__op__. This allows subclasses to ensure an instance of the subclass is returned when interacting with instances of the parent class. - rich comparisons are associated into __op__/__rop__ pairs as follows: __eq__/__eq__ (i.e. a == b is considered equivalent to b == a) __ne__/__ne__ (i.e. a != b is considered equivalent to b != a) __lt__/__gt__ (i.e. a b is considered equivalent to b a) __le__/__ge__ (i.e. a = b is considered equivalent to b = a) - __rpow__ is never invoked for the 3 argument form of pow(), as the coercion rules only apply to binary operations. In this case, a NotImplemented return from the call to __pow__ is converted immediately into a TypeError. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
