Re: [Python-Dev] setuptools: past, present, future
Phillip J. Eby wrote: > At 12:22 AM 4/22/2006 -0400, Terry Reedy wrote: >> Why can't you remove the heuristic and screen-scrape info-search code >>from the easy_install client and run one spider that would check >> new/revised PyPI entries, search for missing info, insert it into PyPI when >> found (and mark the entry eggified), or email the package author or a human >> search volunteer if it does not find enough? > > I actually considered that at one point. After all, I certainly have the > technology. > > However, I didn't consider it for more than 10 seconds or so. Package > authors have no reason to listen to some random guy with a bot -- but they > do have reasons to listen to their users, both actual and potential. I'm not sure that's what Terry meant - I took it to mean *make the spider part of PyPI itself*. So, when you do a PyPI upload, PyPI's spider is triggered, trawls through whatever was uploaded, and adds the results of the search to the PyPI entry for later use by easy_install (e.g under a "Easy Install Info" section - or possibly even a separate page). If there are any problems, PyPI emails the person responsible for the package upload. So any problem reports wouldn't be coming from some random guy with a bot, they'd be coming from PyPI itself. Then all the heuristics and screen-scraping would be server-side - all easy_install would have to do is look at the meta-data provided by the PyPI spider. Cheers, Nick. P.S. I still prefer Py-pee-eye to Cheeseshop. The old name makes even more sense if Phillip's heuristic scan gets added :) -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: improvement of long int and adding new types/modules.
Alex Martelli wrote: >I see "redo Decimal in C" (possibly with the addition of some fast >elementary transcendentals) and "enhance operations on longs" >(multiplication first and foremost, and base-conversions probably >next, as you suggested -- possibly with the addition of some fast >number-theoretical functions) as two separate projects, each of just >about the right magnitude for an SoC project. I would be glad to >mentor either (not both); my preference would be for the former -- it >appears to me that attempting to do both together might negatively >impact both. Remember, it isn't just the coding...: thorough testing, >good docs, accurate performance measurements on a variety of >platforms, ..., are all important part of a project that, assuming >success, it's scheduled to become a core part of Python 2.6, after >all. > > >Alex > > > If it's enough, that's ok for me - I would concentrate on one thing and very likely do it better. My main reason of doing these both at the same time, was that, these would share many code, and since I did quite a bit research in this area (comparing algorithms with diffirent complexity, comparing diffirent cache-friendly implementations, timing on machines starting at p200 ending at dual Xeon 3GHz), I have quite a experience in that stuff. But anyway, conforming to Tim Peters mail, It's not likely something will change in long int ;) - I understand that, and don't want to change you developing python policy (actually, I was expecting that, when I realized that multiply is karatsuba-only). Here is a little comparsion made by my friend python vs my implementation of multiplying (coded in C, without any assembly - I have also assembly version ;P) http://parkour.pl/mat.png It computes product of k*l log(k) = log(l), X axis is sqrt(n), where O(n) = O(logk), in other words, n is 'length' of numbers. I think, that decimal coded in C by me would achieve quite similiar time (also I would eliminate 'staircase' effect), if you wish. I am quite efficiency concerned, in my opinion, efficiency is third most important thing after easy-to-use-code and functionality, and should not be forgetten. Also I state that efficient code/algorithms isn't necessary hard to maintain. And all in all, these algorithms are not so complicated, (in my opinion, fft multiply - which is assymptoticly the best - is less complicated than karatsuba, but that's my opinion). I am now quite sure, what I would like to do, and is possible by you to accept - code decimal in C, most important things about that would: 1.Compatibility with old decimal 1.easy to maintain and expand code 1.don't broke python portability, nor adding new depedencies 2.Efficiency (if and only if 1. isn't broke) I am quite confident I can achieve this points. But everything is open for discussion ;) I'd like to know as much as possible what do you think about this idea. (But already i know quite much) Best regards, Mateusz Rukowicz. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
Phillip J. Eby wrote: > If you have a problem with what I did to the PEP, kindly take it up with > Guido. If you have a problem with the documentation I took the time to > write and contribute, by all means change it. At this point, I'm > getting pretty tired of people of accusing me of violating procedures > around here, and I'm past caring what you do or don't call the bloody > objects. At least I've gotten contextlib and test_contextlib to > actually work, and arranged for there to be *some* documentation for the > "with" statement and the contextlib module. I'm not trying to diminish the work you've done to make this happen - I *did* review those docs after you put them in, and completely missed the discrepancy between them and the wording in the PEP. So the current confusion is at least as much my fault as anyone else's :) The one thing I wasn't sure of after AMK brought it up was whether or not there'd been an offline discussion at PyCon that had made the change on purpose. > Meanwhile, the iterator-iterable analogy is false. You have to be able > to iterate over an iterator, but as AMK pointed out, you don't have to > be able to pass a [thing having __enter__/__exit__] to a "with" > statement. So I was wrong to apply that analogy myself, as are you now. This is only true if we're happy for calling ctx.__context__() explicitly to produce something unusable. i.e., just as these are equivalent: for x in iterable: pass itr = iter(iterable) for x in itr: pass I believe these should be equivalent: with ctx as foo: pass ctx_mgr = ctx.__context__() with ctx_mgr as foo: pass The only way for that to happen is if context managers all have a __context__() method that returns self. > That having been said, I don't think either you or I or even Guido > should be the ones to fix the PEP and the docs at this point, as we've > all stared at the bloody thing way too long to see it with fresh eyes. > So far, AMK is the one who's finding all our screwups, so maybe he > should be the one to explain it all to *us*. :) Heh. I actually had to go trawling back through the python-dev archives to figure out whether or not I was going nuts :) Alternatively, I could have a go at clearing it up for next week's alpha2, and we can ask Anthony to make an explicit request for review of those docs in the announcement. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
On 4/22/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > *This* is what Guido approved, not what is currently written up in the PEP on > python.org. Nick, please get unstuck on the "who said what when and who wasn't listening" thing. I want this to be resolved to use the clearest terminology possible. As you can clearly tell from my recent posts I'm not sure what's best myself. So stop beating people over the head with "Guido approved X". I can't decide this myself -- you and Phillip have to find a way to agree on one version or the other, that's the only pronouncement you will get from me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
Phillip J. Eby wrote: > The problem isn't fundamentally a technical one, but a social one. You can > effect social change through technology, but not by being some random guy > with a nagging 'bot. > Seriously, though, posting Cheesecake scores (which include ratings for > findability of code, use of distutils, etc.) would be a fine way to achieve > the same effect, and if they're part of PyPI itself, they don't give off > the same "random guy with a bot" effect. like "some random bozos who likes play code nazis on the internet" is better than "a random guy with a bot". sheesh. does anyone know if this kind of non-productive control- freakery is common over in Ruby land? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
Terry Reedy wrote: > 1. Based on comments on c.l.py, the biggest legitimate fact-based (versus > personal-taste-based) knock again Python versus, in particular, Perl is the > lack of a CPAN-like facility. As I remember, there have even been a few > people say something like "I like Python the language better that Perl, but > I won't switch because I love CPAN even more." when did anyone last say that? I thought Perl-to-Python migration flame wars was a Y2K thing? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
On 4/22/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Terry Reedy wrote: > > > 1. Based on comments on c.l.py, the biggest legitimate fact-based (versus > > personal-taste-based) knock again Python versus, in particular, Perl is the > > lack of a CPAN-like facility. As I remember, there have even been a few > > people say something like "I like Python the language better that Perl, but > > I won't switch because I love CPAN even more." > > when did anyone last say that? I thought Perl-to-Python migration > flame wars was a Y2K thing? Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be diverse -- yes, I've heard people say that this is something that Python sorely lacks; but I've also heard from more than one person that CPAN sucks from a quality perspective. So I think we shouldn't focus on emulating CPAN; rather, we should solve the problems we actually have. I note that CPAN originated in an age before the web was mature. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
Guido van Rossum wrote: > Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be > diverse -- yes, I've heard people say that this is something that > Python sorely lacks; but I've also heard from more than one person > that CPAN sucks from a quality perspective. So I think we shouldn't > focus on emulating CPAN; rather, we should solve the problems we > actually have. the first problem seems to be to define what those problems really are ;-) (as for the CPAN quality, any public repository will end up being full of crap; I don't see any way to work around that. automatic scoring based on superficial aspects or ratings by small numbers of anonymous visitors are probably among the worst ways to distinguish crap from good stuff; for quality, you need initiatives like http://code.enthought.com/enthon/ and other "fat python" projects. including the standard python.org distribution, of course.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
Phillip J. Eby wrote: > So, if anything is clear from all this, it's that nothing has ever been > particularly clear in all this. :) > > Or more precisely, I think everybody has been perfectly clear, we just > haven't really gotten on the same page about which words mean what. ;) +1 QOTT (Quote of the Thread) :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in 2.5.
Ian Bicking wrote: > For instance, if you really want to be confident about how your libraries > are layed out, this script is the most reliable way: > http://peak.telecommunity.com/dist/virtual-python.py note the use of "this script is the most reliable way", not "something like this script", or "you have to do this, see e.g." it's pretty easy to get the feeling that you and Phillip seem to think that you're the only ones who have ever addressed these problems, and that your solutions are automatically superior to anyone elses... (frankly, do you think there's any experienced developer out there whos first thought when asked the question "how do I create a tightly controlled Python environment" isn't either "can I solve this by tweaking sys.path in my application?" or "disk space is cheap, bugs are expensive; let's use a separate install", spends 15 minutes setting that up, checks in the result, and goes back to working on the hard stuff...) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 -peps/trunk/pep-0343.txt)
Guido van Rossum wrote: > Nick, please get unstuck on the "who said what when and who wasn't > listening" thing. I want this to be resolved to use the clearest > terminology possible. which probably means that the words "context" and "manager" shouldn't be used at all ;-) "space" and "potato", perhaps? like in http://tinyurl.com/k5spk ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
On 4/22/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Alternatively, I could have a go at clearing it up for next week's alpha2, and
> we can ask Anthony to make an explicit request for review of those docs in the
> announcement. . .
I've just had a *very* quick scan through the 2.5a1 documentation. I
did not look at the PEP, just the official documentation. I've been
reading the messages going round on the subject, but I'm getting
pretty confused, so I'd still count myself as "unprejudiced"... :-)
My immediate reaction was that the docs make reasonable sense:
- a context is a thing with enter/exit methods (a block of code is
"in" a context)
- the with statement delimits the block which is in a context
- the with statement asks a context manager for the context in which
the block runs
- context managers have __context__ methods to produce contexts (they
manage the production of explicit context objects)
The contextlib.contextmanager decorator starts off looking fine as well:
@contextmanager
def tag(name):
print "<%s>" % name
yield
print "" % name
Yes, that's a context manager - you pass it to a with statement:
>>> with tag("h1"):
...print "foo"
...
foo
But then things go wrong:
class Tag:
def __init__(self, name):
self.name = name
@contextmanager
def __context__(self):
print "<%s>" % self.name
yield self
print "" % self.name
h1 = Tag("h1")
That's bizarre: __context__ isn't the context manager I'm trying to
create - those are the instances of Tag. I think this is where the
terminology comes unstuck, and it's simply because this is an "abuse"
(a bit strong, that - bear with me) of the contextmanager decorator.
The thing is, __context__ should be *a function which returns a
context*. But we defined it with the decorator as a context manager -
an object whose __context__ method produces a context! It works,
because context managers produced by the decorator return themselves -
that is, they are both context managers and contexts... No, I just
got lost.
BUT - the point is that everything was fine until the point where the
__context__ method got defined using @contextmanager. Maybe all we
need is to have *two* decorators - @context to generate a context
(this would be @contextmanager but without the __context__ method) and
@contextmanager as now (actually, it only needs the __context__ method
- the __enter__ and __exit__ methods are only present to allow the
trick of returning self from __context__).
Then, the definitions are easy:
context manager - has __context__ producing a context
context - has __enter__ and __exit__ methods, used by the with statement
Things with all 3 methods are just a convenience trick to avoid
defining 2 objects - there's no *need* for them (unlike iterators,
where "iter(it) is it" is an important defining characteristic of an
iterator over an iterable).
So my proposal:
- use the definitions above
- modify contextlib to have 2 decorators - @contextmanager producing a
context manager, and @context producing a context. They can be the
same under the hood, using an object that defines all 3 methods, but
that's just an implementation detail (trick)
- amend the documentation of the Tag example in the contextlib docs to
use the @context decorator.
- tidy up the PEP to reflect this approach
Or alternatively, I'm just confused, like the rest of you :-)
Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 -peps/trunk/pep-0343.txt)
Fredrik Lundh wrote: > Guido van Rossum wrote: > >> Nick, please get unstuck on the "who said what when and who wasn't >> listening" thing. Sorry about that. I was just trying to figure out how we got to where we are. I stopped paying close attention to PEP 343 developments a few months back, and ended up catching up out loud here on the list. . . > I want this to be resolved to use the clearest >> terminology possible. I'm planning to have one go at it before next week's 2nd alpha (making sure the source code, library reference, language reference and PEP are all at least superficially consistent), and then asking Anthony to include something in the 2nd alpha announcement explicitly requesting review of these docs. As Phillip pointed out, we need input from people that haven't been intimately involved in the PEP 343 discussions to see if the final docs actually make sense. As I discovered in reviewing the contextlib docs, it turned out to be awfully easy for me to see what I expected to see rather than what was actually there. > which probably means that the words "context" and "manager" shouldn't > be used at all ;-) > > "space" and "potato", perhaps? > > like in http://tinyurl.com/k5spk ? That's beautiful. It even matches the PEP [1] :) Cheers, Nick. [1] http://tinyurl.com/pc5uq -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] New artwork for the osx port
Hi, Over on the pythonmac-sig list we're getting close a new set of icons based on the new python.org logo. What would be needed to get these icons into the python distribution? Does the author of these icons need to donate them to the PSF or is there some other procedure? Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
Paul Moore wrote: > On 4/22/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: >> Alternatively, I could have a go at clearing it up for next week's alpha2, >> and >> we can ask Anthony to make an explicit request for review of those docs in >> the >> announcement. . . > > I've just had a *very* quick scan through the 2.5a1 documentation. I > did not look at the PEP, just the official documentation. I've been > reading the messages going round on the subject, but I'm getting > pretty confused, so I'd still count myself as "unprejudiced"... :-) Thanks for doing that. I got lost in a maze of twisty contexts all alike around the same place you did, so we apparently need to do something different somewhere. So I'm going to express my gratitude by asking you to read the same docs all over again in a few days time :) > My immediate reaction was that the docs make reasonable sense: > > - a context is a thing with enter/exit methods (a block of code is > "in" a context) > - the with statement delimits the block which is in a context > - the with statement asks a context manager for the context in which > the block runs > - context managers have __context__ methods to produce contexts (they > manage the production of explicit context objects) I'll be making a pass through the docs (and PEP) this weekend using the definitions: - a context manager is a thing with enter/exit methods (it sets up and tears down an execution context for a block of code) - the with statement delimits the block which is in an execution context - the with statement asks a context object for a context manager to set up and tear down an execution context when the block runs - context objects have a __context__ method to produce context managers (hey, it isn't really that much worse than using the __iter__ method to ask an iterable for an iterator. . .) I'll also add something in which parallels the current "Iterator Types" section in the library reference (only for "Context Types"). The big changes from where we are currently are that: - "execution context" will be introduced for the sundry effects that a context manager may have on the code in the body of a with statement (like affecting how exceptions are handled, redirecting IO, changing the thread's active decimal context, affecting thread synchronisation etc) - "context object" will be used where "context manager" is currently used. This is mainly so that decimal.Context can be safely referred to as being a context object. - "context manager" will be used where "context" is currently used. This is so that the __context__ method returns context managers, which means decorating the generator based ones with @contextlib.contextmanager makes sense. I was considering producing a patch instead so it could be reviewed before I changed anything, but I don't think we'll really understand which is clearer until we can review it all together, and documentation patches are difficult to review properly without applying them and rebuilding the docs (which a lot of people aren't set up to do - just ask the effbot ;). If the terminology *still* breaks down with those slightly different definitions, we'll have to try to come up with a third option after the 2nd alpha. I'm really hoping my planned changes actually work out, because if they don't I'm out of ideas for how to make these concepts easier to grok. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
On 4/22/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > So I'm going to express my gratitude by asking you to read the same docs all > over again in a few days time :) No problem. Remind me if I forget... > I'll be making a pass through the docs (and PEP) this weekend using the > definitions: > >- a context manager is a thing with enter/exit methods > (it sets up and tears down an execution context for a block of code) >- the with statement delimits the block which is in an execution context >- the with statement asks a context object for a context manager to set up > and tear down an execution context when the block runs >- context objects have a __context__ method to produce context managers > (hey, it isn't really that much worse than using the __iter__ method to > ask an iterable for an iterator. . .) Sorry, but I don't really like this. I find the idea of a context manager, creating contexts, within which the block in a with statement runs, much more intuitive. As I said, the only issue I have with it is the dual use of the contextmanager decorator (and I think that's fundamental - there are 2 different things going on, and they *should* have different names). But I'll do my best to put away my prejudices and read the new docs as they are written, when they come out. > If the terminology *still* breaks down with those slightly different > definitions, we'll have to try to come up with a third option after the 2nd > alpha. I'm really hoping my planned changes actually work out, because if they > don't I'm out of ideas for how to make these concepts easier to grok. . . Presumably, then, my proposal didn't make things clear to you? I won't comment further on your proposal, as I *want* to avoid thinking about it before I read the docs... Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
Paul Moore wrote: > Presumably, then, my proposal didn't make things clear to you? As Phillip said, I'm probably way too close to this to be a good judge of how understandable the terminology is. I just want to make one more attempt before admitting defeat. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New artwork for the osx port
On 4/22/06, Ronald Oussoren <[EMAIL PROTECTED]> wrote: > Over on the pythonmac-sig list we're getting close a new set of icons > based on the new python.org logo. What would be needed to get these > icons into the python distribution? Does the author of these icons > need to donate them to the PSF or is there some other procedure? I guess the better place to ask is [EMAIL PROTECTED] -- this reaches the PSF board which decides and has expertice about such matters. (And no, I'm not on that list any more -- I've learned to delegate. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
On Sat, 22 Apr 2006, Fredrik Lundh wrote: > Guido van Rossum wrote: [...] >> Python sorely lacks; but I've also heard from more than one person >> that CPAN sucks from a quality perspective. So I think we shouldn't [...] > (as for the CPAN quality, any public repository will end up being full > of crap; I don't see any way to work around that. automatic scoring [...] I had assumed Guido was referring to the quality of the infrastructure, including CPAN.pm, rather than the quality of the code stored in CPAN. I've certainly heard at least two people complain about the usability and reliability of the CPAN infrastructure recently, and recall I found it rather unfriendly myself. But that was around 5 years ago; I may simply be wrong or out of date. John ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
I was actually referring to the quality of the code. On 4/22/06, John J Lee <[EMAIL PROTECTED]> wrote: > On Sat, 22 Apr 2006, Fredrik Lundh wrote: > > Guido van Rossum wrote: > [...] > >> Python sorely lacks; but I've also heard from more than one person > >> that CPAN sucks from a quality perspective. So I think we shouldn't > [...] > > (as for the CPAN quality, any public repository will end up being full > > of crap; I don't see any way to work around that. automatic scoring > [...] > > I had assumed Guido was referring to the quality of the infrastructure, > including CPAN.pm, rather than the quality of the code stored in CPAN. > > I've certainly heard at least two people complain about the usability and > reliability of the CPAN infrastructure recently, and recall I found it > rather unfriendly myself. But that was around 5 years ago; I may simply > be wrong or out of date. > > > John > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
On 20-apr-2006, at 23:46, Martin v. Löwis wrote: > Bob Ippolito wrote: >>> 'There are several binary formats that embody eggs, but the most >>> common >>> is '.egg' zipfile format, because it's a convenient one for >>> distributing >>> projects.' >>> >>> '.egg files are a "zero installation" format for a Python package;' >> >> single modules are also such a "zero installation" format too. So >> what? >> >> You're simply reading things between the lines that aren't there. >> How >> about you describe exactly what parts of the documentation that >> lead you >> to believe that eggs are designed to compete with solutions like >> rpm/msi/deb so that it can be clarified? > > It's not just the documentation: I firmly believe that many people > consider .egg files to be a distribution and package management > format. People have commented that some systems (e.g. OSX) doesn't > have a usable native packager, so setuptools fills a need here. > This shows that people do believe that .egg files are to OSX what > .deb files are to Debian. As .egg files work on Debian, too, > it is natural that they compete with .deb. > > Phillip Eby once said (IIRC) that he doesn't want package authors to > learn all the different bdist_* commands (which also require access > to the target systems sometimes), and that they their life gets easier > as they now only have to ship the "native" Python binary packages, > namely .egg files. > > In this view, rpm/msi/deb have no place anymore, and are obsolete. In the view of at least some Linux packagers nobody but they should create system packages anyway. Personally I think that view is misguided, but the view is there. > > I can readily believe that package authors indeed see this as > a simplification, but I also see an increased burden on system > admins in return. > > So if this attitude (Python Eggs are the preferred binary distribution > format) is wrong, it is the attitude that has to change first. Changes > to the documentation follow from that. If the attitude is right, I'll > have to accept that I have a minority opinion. IMHO python eggs are the preferred distribution format for several use cases, but not all. They are very usefull for systems that lack a proper package manager of their own and for managing a developers sandbox. As a sysadminI'd be a lot less inclined to use eggs to install software on a system with a proper package manager (like most linux distributions) because the eggs will then not be visible in the global view of installed software or play nice with vendor software management tools. Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
At 12:34 PM 4/22/2006 +0200, Fredrik Lundh wrote: >Guido van Rossum wrote: > > > Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be > > diverse -- yes, I've heard people say that this is something that > > Python sorely lacks; but I've also heard from more than one person > > that CPAN sucks from a quality perspective. So I think we shouldn't > > focus on emulating CPAN; rather, we should solve the problems we > > actually have. > >the first problem seems to be to define what those problems really >are ;-) > >(as for the CPAN quality, any public repository will end up being full >of crap; I don't see any way to work around that. automatic scoring >based on superficial aspects The purpose of automated scoring on superficial aspects isn't so much to ensure quality as it is to ensure *accessibility*, both in the sense of being able to install the thing, and meet some basic levels of having documentation. If something is accessible and trivial to install, then the market can decide which packages are better to actually use. >or ratings by small numbers of anonymous >visitors are probably among the worst ways to distinguish crap from >good stuff; for quality, you need initiatives like > > http://code.enthought.com/enthon/ > >and other "fat python" projects. Actually, every project that uses other projects' code can now be a "chubby python" by expressing dependencies. Really, one of the best ratings of a package's quality (or at least popularity) is going to be how many other projects depend on it. If everybody uploaded eggs to the Cheeseshop, it'd be possible to show links to "projects that use this project's code" by reading the dependency metadata with pkg_resources. (Not to mention "projects that this project uses"). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] With context, please
I've been following the with/context discussion, not that closely, but reading all the posts. I also have to write docs on this for Python for Dummies, which I think is going to be the first book out after 2.5. So far, my take is that I want the block of code to be executed in a context. I'm probably going to use that terminology no matter what gets decided here -- I think it's the only sensible way to describe it for newcomers. Aside from that, I don't care all that much. (Actually, we just turned in the first draft, and I haven't talked about context managers at all -- what I said was that EXPR returns a context.) -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] With context, please
At 09:05 AM 4/22/2006 -0700, Aahz wrote: >I've been following the with/context discussion, not that closely, but >reading all the posts. I also have to write docs on this for Python for >Dummies, which I think is going to be the first book out after 2.5. So >far, my take is that I want the block of code to be executed in a >context. I'm probably going to use that terminology no matter what gets >decided here -- I think it's the only sensible way to describe it for >newcomers. Aside from that, I don't care all that much. > >(Actually, we just turned in the first draft, and I haven't talked about >context managers at all -- what I said was that EXPR returns a context.) And what did you say that __context__ returns? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in 2.5.
At 12:34 PM 4/22/2006 +0200, Fredrik Lundh wrote: >Ian Bicking wrote: > > > For instance, if you really want to be confident about how your libraries > > are layed out, this script is the most reliable way: > > http://peak.telecommunity.com/dist/virtual-python.py > >note the use of "this script is the most reliable way", not "something >like this script", or "you have to do this, see e.g." Picky, picky, picky. As it happens, EasyInstall's documentation used to just explain the steps, and people would complain about how hard it was. Ian wrote a script to do it automatically, and I touched it up a bit for distribution. While I personally wouldn't have said it the same way Ian did, there is nonetheless a point to his saying it in that way. If you are giving people help, you don't give ambiguous recommendations. More to the point, you don't tell somebody to reinvent something that already exists. If they were the reinventing type, they'd have already read the documentation and either decided to use the tool or not, to tweak it or not, etc., on their own, rather than asking on a mailing list for help. So your projection of attitudes here has nothing to do with Ian. >(frankly, do you think there's any experienced developer out there >whos first thought when asked the question "how do I create a tightly >controlled Python environment" isn't either "can I solve this by tweaking >sys.path in my application?" or "disk space is cheap, bugs are expensive; >let's use a separate install", spends 15 minutes setting that up, checks >in the result, and goes back to working on the hard stuff...) Clearly, we're not dealing with "experienced" developers, then. Of course even now that it's easy to *do*, some people still gripe that setting up a separate install is too "heavy". (Except the audience for whom the script was intended, who consider it a godsend.) Some people are never satisfied, obviously. Anyway, while we're projecting about people's attitudes, what's with this "Real Programmers Should Build It Themselves" attitude? What are you, some kind of Lisp programmer? ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] With context, please
On Sat, Apr 22, 2006, Phillip J. Eby wrote: > At 09:05 AM 4/22/2006 -0700, Aahz wrote: >> >>I've been following the with/context discussion, not that closely, but >>reading all the posts. I also have to write docs on this for Python for >>Dummies, which I think is going to be the first book out after 2.5. So >>far, my take is that I want the block of code to be executed in a >>context. I'm probably going to use that terminology no matter what gets >>decided here -- I think it's the only sensible way to describe it for >>newcomers. Aside from that, I don't care all that much. >> >>(Actually, we just turned in the first draft, and I haven't talked about >>context managers at all -- what I said was that EXPR returns a context.) > > And what did you say that __context__ returns? Whoops, I half-lied. I forgot that my co-author did indeed mention "context manager". Here's the main part (sorry about the missing formatting): The syntax is as follows: with EXPRESSION as NAME: BLOCK The with statement works like this: EXPRESSION returns a value that the with statement uses to create a context (a special kind of namespace). The context is used to execute the BLOCK. The block might end normally, get terminated by a break or return, or raise an exception. No matter which of those things happens, the context contains code to clean up after the block. The as NAME part is optional. If you include it, you can use NAME in your BLOCK Then a bit later: The protocol used by the with statement is called the context management protocol, and objects implementing it are context managers. However, we do not talk at all about __context__(), __enter__(), or __exit__() -- we decided that was too advanced for our audience, we simply wanted to give them the bare basics of using `with`. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] With context, please
At 09:25 AM 4/22/2006 -0700, Aahz wrote: > EXPRESSION returns a value that the with statement uses to create a > context (a special kind of namespace). The context is used to > execute the BLOCK. The block might end normally, get terminated by > a break or return, or raise an exception. No matter which of those > things happens, the context contains code to clean up after the > block. > > The as NAME part is optional. If you include it, you can use NAME > in your BLOCK > >Then a bit later: > > The protocol used by the with statement is called the context > management protocol, and objects implementing it are context > managers. Okay, which means that you agree with AMK and Paul Moore that the thing you pass to "with" is a context manager, and the thing that controls execution is a context. Was that conclusion independently arrived at, or based on reading e.g. the docs I wrote? Obviously, if you guys came up with that terminology on your own, that's a stronger vote in its favor. Btw, the phrase "special kind of namespace" seems wrong to me, since there are no names in a context, and that phrase makes it sound like you get a new scope. Looks to me like you could replace the word "namespace" with "object" without changing the intended effect. (That is, I assume the intended effect was merely to point out you're introducing a new term that the reader is not yet expected to know.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
At 05:41 PM 4/22/2006 +1000, Nick Coghlan wrote: >Phillip J. Eby wrote: >>At 12:22 AM 4/22/2006 -0400, Terry Reedy wrote: >>>Why can't you remove the heuristic and screen-scrape info-search code >>>from the easy_install client and run one spider that would check >>>new/revised PyPI entries, search for missing info, insert it into PyPI when >>>found (and mark the entry eggified), or email the package author or a human >>>search volunteer if it does not find enough? >>I actually considered that at one point. After all, I certainly have the >>technology. >>However, I didn't consider it for more than 10 seconds or so. Package >>authors have no reason to listen to some random guy with a bot -- but >>they do have reasons to listen to their users, both actual and potential. > >I'm not sure that's what Terry meant - I took it to mean *make the spider >part of PyPI itself*. Which would also be accomplished by using Grig's Cheesecake tool, since it uses easy_install to fetch the source. >Then all the heuristics and screen-scraping would be server-side - all >easy_install would have to do is look at the meta-data provided by the >PyPI spider. Which is certainly attractive from the POV of being able to make changes quickly. However, I forgot to mention another issue, because I was speaking from the point of view of the time when I designed the thing, not the present day. After it was implemented, it has turned out that being able to point easy_install to web pages with a specific collection of packages (e.g. ones built for a specific OS version, or that are tested for a particular purpose, etc.) is *very* useful in practice. And the people who are doing that, are just going to do whatever it takes to make their listing(s) work with easy_install, because that's the whole point for them. So there doesn't have to be unlimited growth of heuristics there. What it basically amounts to, then, is that easy_install heuristics currently only have to chase people who aren't trying to easy_install their packages. For example, I discovered the other day that easy_install can get confused by bdist_dumb distributions. So few people ever distribute bdist_dumb packages that I never ran into that as an issue before now. So I had to update the heuristics to be able to tell from the filename whether a package is likely to be a bdist_dumb. However, if PyPI is doing Cheesecake ratings, there will only be a finite number of such things to deal with, because when people make changes that break their ratings, they'll just fix the problem themselves, as it'll generally be faster than lobbying for new heuristics in easy_install. As the community becomes better educated about making their package links easy to find, the amount of maintenance work needed for easy_install should drop off. Right now, the main reason to add heuristics is to increase compatibility with whatever practices are already out there, in order to leverage the greatest number of existing packages to secure the greatest number of users. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in 2.5.
Phillip J. Eby wrote: > >(frankly, do you think there's any experienced developer out there > >whos first thought when asked the question "how do I create a tightly > >controlled Python environment" isn't either "can I solve this by tweaking > >sys.path in my application?" or "disk space is cheap, bugs are expensive; > >let's use a separate install", spends 15 minutes setting that up, checks > >in the result, and goes back to working on the hard stuff...) > > Clearly, we're not dealing with "experienced" developers, then. the paragraph you're quoting used "experienced" in the context of "having solved this problem before". > Of course even now that it's easy to *do*, some people still gripe that > setting up a separate install is too "heavy". what forums are we talking about here? (if this kind of complaints were common on c.l.python, for example, I think I would have noticed...) > Anyway, while we're projecting about people's attitudes, what's with this > "Real Programmers Should Build It Themselves" attitude? because that's what Python is all about: making things so easy that every- one can build things with a minimum of effort, according to their specific requirements. "a lot of action in a small amount of clear code", as some- one once put it. that doesn't rule out helpful libraries and utilities and cookbook examples, but a "we have a prepackaged solution for you, it's the only solution you'll ever need, and you don't really need to know how it works" approach doesn't strike me as very Pythonic. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
Guido van Rossum wrote: > Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be > diverse -- yes, I've heard people say that this is something that > Python sorely lacks; but I've also heard from more than one person > that CPAN sucks from a quality perspective. So I think we shouldn't > focus on emulating CPAN; rather, we should solve the problems we > actually have. I note that CPAN originated in an age before the web > was mature. My personal problems with CPAN were always of the kind that it recorded too many/too stringent dependencies. I used it over a period of several years on Solaris, roughly two times a year. Each time, the package I wanted to installed depended on another package, this in turn on a third, and some of these eventually on a Perl version more recent than the one I had installed. So CPAN would always *first* install a new version of Perl for me. Sometimes, this would fail, because Perl wouldn't pass its test suite on Solaris. So I did huge downloads, long compilation times, and still didn't get the package installed. I always fixed it by installing the new Perl version manually, and then starting over with CPAN again. I'm not exactly sure why that happened, but I think there are two causes: - when installing a package, the automated download tool should not try to find the most recent version. Instead, it should try to find a version that causes the least amount of changes to my system. - CPAN shouldn't include Perl proper (likewise, the Cheesehop shouldn't include Python proper). If dependencies can't be resolved with the current version, but could be resolved with a later version, the download tool should give up and explain it all. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding wsgiref
What with all the discussion that resulted from setuptools, we should
probably also discuss the suggestion to add wsgiref to the standard
library. PEP 356 doesn't have many details about what's under consideration.
(wsgiref is an implementation of the WSGI interface defined in PEP 333.
I believe the latest version is at
.)
I expect that most of the package would be added. wsgiref.handlers is
the heart of it, and needs wsgiref.{headers,util}.
wsgiref.simple_server might be debatable; the module docstring warns
that the code hasn't been reviewed for security issues, but on the
other hand if there's a WSGI library, we do want the available HTTP
server to support it.
--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding wsgiref
On 4/22/06, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
> What with all the discussion that resulted from setuptools, we should
> probably also discuss the suggestion to add wsgiref to the standard
> library. PEP 356 doesn't have many details about what's under consideration.
>
> (wsgiref is an implementation of the WSGI interface defined in PEP 333.
> I believe the latest version is at
> .)
>
> I expect that most of the package would be added. wsgiref.handlers is
> the heart of it, and needs wsgiref.{headers,util}.
> wsgiref.simple_server might be debatable; the module docstring warns
> that the code hasn't been reviewed for security issues, but on the
> other hand if there's a WSGI library, we do want the available HTTP
> server to support it.
I'd like simple_server; I've got an app based on it already.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (wasr45544 -peps/trunk/pep-0343.txt)
"Nick Coghlan" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > As Phillip pointed out, we need input from people that haven't been > intimately > involved in the PEP 343 discussions OK, here is my attempt to cut the knot. To me, 'context' and 'context manager' can be seen as near synonyms; either could be used to describe the thing that 'governs' the block execution. I (and some other others) prefer the shorter term; yet I can see how someone (you, at least) could prefer the longer, more explicit term. To me, the thing after 'with' that makes the whatever for the block is DEFINITELY not a 'context'; trying to twist context to mean that is a brain twister. Calling it 'context manager' is possible if one interpretes 'manager' instead as a hands-off manager who appoints a foreman to do the actual work and then departs. But the term is ambiguous as this discussion has shown. So I propose that the context maker be called just that: 'context maker'. That should pretty clearly not be the context that manages the block execution. An additional source of confusion is that we can name a function got several reasons; among them one is what it is, another is what it returns. For instance, a generic generator for-loop could be written as either of for item in generator_function(): for item in generator(): In context, I think the second reads better, as long as it is clear that the function name 'generator' refers what it returns and not what it is. Similar, a context_maker function could be named any of 'context_maker', 'context_manager', or 'context', with the latter two referring to the return value. In the context of 'with as name:', either of the latter two reads better to me. I would call the decorator @contextmaker since that is what it turns the decorated function into. Well, I hope this slightly different viewpoint is at least a bit helpful. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
"Guido van Rossum" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be > diverse -- yes, I've heard people say that this is something that > Python sorely lacks; but I've also heard from more than one person > that CPAN sucks from a quality perspective. So I think we shouldn't > focus on emulating CPAN; No, we should aim to do better, both in terms of functionality, if that is possible, and contents. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
"Phillip J. Eby" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > At 05:41 PM 4/22/2006 +1000, Nick Coghlan wrote: >>I'm not sure that's what Terry meant - I took it to mean *make the spider >>part of PyPI itself*. > > Which would also be accomplished by using Grig's Cheesecake tool, since > it > uses easy_install to fetch the source. I think Nick was much closer to what I meant. Let me try again. As I understood your post, setuptools/easyinstall has some spider, heuristic, and screen-scrape code that tries to fetch info that one would like to have been in PyPI, but is not. I inferred that if the fetched info is not cached anywhere, then mutiple clients would have to repeat the process. Based on this understanding, and cognizant that your project's newly elevated status opens options that you did not have before, I had three related suggestions: 1. Move appropriate code from all the clients to one server, either associated with the PyPI server or even that server itself. (Among other things, this would allow you to update heuristics, etc, without distribution to existing clients or worry about bloating them.) 2. Once missing info is discovered, save it so the discovery process is not repeated. 3. If the search fails, email *someone*. I suggested *either* the package author (under an authoritative signature) or a non-author volunteer who could proceed somehow, such as searching more or contacting the author as a human. If my premises above are mistaken, then the suggestions should be modified or discarded. However, I don't see how they conflict at all with a consumer rating system. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
At 08:12 PM 4/22/2006 -0400, Terry Reedy wrote: >If my premises above are mistaken, then the suggestions should be modified >or discarded. However, I don't see how they conflict at all with a >consumer rating system. My point was simply that providing rapid, visible feedback to authors results in a shorter feedback loop with less infrastructure. Also, after thinking it over, it's clear that the spidering is never going to be able to come out entirely, because there are lots of valid use cases for people effectively setting up their own mini-indexes. All that will happen is that at some point I'll be able to stop adding heuristics. (Hopefully that point is already past, in fact.) For anybody that wants to know how the current heuristics work, EasyInstall actually only has a few main categories of heuristics used to find packages: * Ones that apply to PyPI * Ones that apply to SourceForge * Ones that interpret distutils-generated filenames * The one that detects when a page is really a Subversion directory, and thus should be checked out instead of downloaded Most of the SourceForge heuristics have been eliminated already, except for the translation of prdownloads.sf.net URLs to dl.sourceforge.net URLs, and automatic retries in the event of a broken mirror. I'm about to begin modifying the PyPI heuristics to use the new XML-RPC interface instead, for the most part. (Although finding links in a package's long description will still be easier via the web interface.) And the distutils haven't started generating any new kinds of filenames lately, although I occasionally run into situations where non-distutils links or obscure corner cases of distutils filenames give problems, or where somebody has filenames that look like they came from the distutils, but the contents aren't a valid distutils source distribution. Anyway, these are the only things that are truly heuristic in the sense that they are attempts to guess well, and there is always the potential for failure or obsolescence if PyPI or SourceForge or Subversion changes, or people do strange things with their links. I should probably also point out that calling this "spidering" may give the impression it's more sophisticated than it is. EasyInstall only retrieves pages that it is explicitly given, or which appear in one of two specific parts of a PyPI listing. But it *scans* links on any page that it retrieves, and if a link looks like a downloadable package, it will parse as much info as practical from the filename in order to catalog it as a possible download source. So, it will read HTML from PyPI pages, pages directly linked from PyPI as either "Home" or "Download" URLs, and page URLs you give to --find-links. But it doesn't "spider" anywhere besides those pages, unless you count downloading an actual package link. The whole process more resembles a few quick redirects in a browser, than it does any sort of traditional web spider. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Ronald Oussoren wrote: > On 20-apr-2006, at 23:46, Martin v. Löwis wrote: >> So if this attitude (Python Eggs are the preferred binary distribution >> format) is wrong, it is the attitude that has to change first. Changes >> to the documentation follow from that. If the attitude is right, I'll >> have to accept that I have a minority opinion. > > IMHO python eggs are the preferred distribution format for several > use cases, but not all. They are very usefull for systems that lack a > proper package > manager of their own and for managing a developers sandbox. > > As a sysadminI'd be a lot less inclined to use eggs to install > software on a system with a proper package manager (like most linux > distributions) because the eggs will then not be visible in the > global view of installed software or play nice with vendor software > management tools. Maybe we need something that's the equivalent of alien (rpm -> dpkg converter), so that given an egg, one can easily get a native installer for that egg. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (1)
[19 Apr 2006, Neal Norwitz] > test_cmd_line leaked [0, 17, -17] references > test_filecmp leaked [0, 13, 0] references > test_threading_local leaked [-93, 0, 0] references > test_urllib2 leaked [-121, 88, 99] references Thanks to Thomas digging into test_threading_local, I checked in what appeared to be a total leak fix for it last week. On my Windows box, it's steady as a rock now: """ $ python_d -E -tt ../lib/test/regrtest.py -R:50: test_threading_local test_threading_local beginning 55 repetitions 1234567890123456789012345678901234567890123456789012345 ... 1 test OK. [27145 refs] """ Is it still flaky on other platforms? If not, maybe the reported test_threading_local leaked [-93, 0, 0] references is due to stuff from a _previous_ test getting cleaned up (later than expected/hoped)? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
[Andrew MacIntyre] > I doubt it has anything to do with this issue, but I just thought I'd > mention something strange I've encountered on Windows XP Pro (SP2) at > work. > > If Python terminates due to an uncaught exception, with stdout & stderr > redirected externally (ie within the batch file that started Python), What batch file? > the files that were redirected to cannot be deleted/renamed until the > system is rebooted. This really needs an executable example. Here's my best guess about what you mean, but I don't see any of what you're describing on WinXP Pro SP2: """ $ type batty.py import sys i = 0 for line in sys.stdin: sys.stdout.write(line) i += 1 if i == 3: raise "uncaught exception" $ type runpy.bat @python batty.py < stdin.txt > stdout.txt 2>stderr.txt $ type stdin.txt a b c d e f g h i $ runpy.bat $ type stdout.txt a b c $ type stderr.txt batty.py:8: DeprecationWarning: raising a string exception is deprecated raise "uncaught exception" Traceback (most recent call last): File "batty.py", line 8, in raise "uncaught exception" uncaught exception $ del stdout.txt $ del stderr.txt $ dir/b std*.txt stdin.txt """ I'll note that stdin.txt can also be deleted when runpy.bat finishes. > If a bare except is used to trap any such exceptions (and the traceback > printed explicitly) so that Python terminates normally, there is no > problem (ie the redirected files can be deleted/renamed etc). > > I've never reported this as a Python bug If you do, I'll probably add a comment like the above ;-) > because I've considered the antivirus SW likely to be the culprit. Could be. FWIW, Norton AV was running during the above. > I don't recall seeing this with Windows 2000, but much was changed > in the transition from the Win2k SOE to the WinXP SOE. What's that? Shitty Out-of-box Experience ;-)? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
On Sunday 23 April 2006 11:43, Nick Coghlan wrote: > Maybe we need something that's the equivalent of alien (rpm -> dpkg > converter), so that given an egg, one can easily get a native > installer for that egg. An 'eggconvert' that used the existing bdist_foo machinery to build system specific packages would probably be an ok summer of code project, no? -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
At 01:19 PM 4/23/2006 +1000, Anthony Baxter wrote: >On Sunday 23 April 2006 11:43, Nick Coghlan wrote: > > Maybe we need something that's the equivalent of alien (rpm -> dpkg > > converter), so that given an egg, one can easily get a native > > installer for that egg. > >An 'eggconvert' that used the existing bdist_foo machinery to build >system specific packages would probably be an ok summer of code >project, no? That's probably not going to be the best way to get from an egg to a system package, since a lot of the bdist_foo commands try to build things from source, and an egg for the specific platform is already going to be built, and won't include source (except for Python modules). Probably you'd want to create something more like Vincenzo Di Massa's "easy_deb" program, which uses easy_install to find and fetch a source distribution, then builds a .deb from it. You can currently use: easy_install --editable --build_directory=somewhere SomePackage And it will find SomePackage, and unpack a source distribution into 'somewhere/somepackage'. So you could then change to that directory and run the package's setup.py with any bdist command you wanted to. So, for any bdist_foo command that's already implemented in the distutils, you can already get pretty close to this functionality by using a short shell script that just calls easy_install followed by the setup.py. What you won't get without writing some more code is dependency support. You also have to deal with the issue of mapping PyPI names to the names used by the relevant packaging system. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] With context, please
Phillip J. Eby wrote: > At 09:25 AM 4/22/2006 -0700, Aahz wrote: >> EXPRESSION returns a value that the with statement uses to create a >> context (a special kind of namespace). The context is used to >> execute the BLOCK. The block might end normally, get terminated by >> a break or return, or raise an exception. No matter which of those >> things happens, the context contains code to clean up after the >> block. >> >> The as NAME part is optional. If you include it, you can use NAME >> in your BLOCK >> >> Then a bit later: >> >> The protocol used by the with statement is called the context >> management protocol, and objects implementing it are context >> managers. > > Okay, which means that you agree with AMK and Paul Moore that the thing you > pass to "with" is a context manager, and the thing that controls execution > is a context. Was that conclusion independently arrived at, or based on > reading e.g. the docs I wrote? Obviously, if you guys came up with that > terminology on your own, that's a stronger vote in its favor. I think I've figured out where you and I went off in different directions with this - when you read "context management protocol" in the PEP you interpreted it as "has a __context__ method that produces an object with __enter__/__exit__ methods", but when I originally added the term "context management protocol" to the PEP what I actually meant was "has __enter__/__exit__ methods and a __context__ method that returns self" (the last part of that definition being added implicitly when the __context__ method was introduced). Starting from that point, I'm no longer surprised you considered the PEP to be inconsistent in its use of the terminology :) As far as I can tell, Aahz's book doesn't currently say anything that favours one interpretation over the other (which is probably a good thing from Aahz's point of view :). In case its not entirely clear why I think Aahz's wording is neutral, it's because in my intended interpretation the context manager sets up and tears down an abstract execution context which is distinct from the concrete context object that provided the manager in the first place. The manager is a mediator that translates from the data passively recorded in the context object to the appropriate active manipulations of the runtime state. I realise the overloading of the term 'context' is potentially confusing, and I didn't clearly acknowledge the distinction myself until the recent discussion. Not acknowledging that distinction appears to have been largely responsible for my manifest failure to document this properly in the PEP. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Anthony Baxter wrote: > On Sunday 23 April 2006 11:43, Nick Coghlan wrote: >> Maybe we need something that's the equivalent of alien (rpm -> dpkg >> converter), so that given an egg, one can easily get a native >> installer for that egg. > > An 'eggconvert' that used the existing bdist_foo machinery to build > system specific packages would probably be an ok summer of code > project, no? It sounds like an excellent one if you can find the right mentor. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Reducing memory overhead for dictionaries by removing me_hash
Hi, this is my first python dev post, so please forgive me if this topic has already been discussed.It seemed to me that removing me_hash from a dict entry would save 2/3 of the space used by dictionaries and also improve alignment of the entries since they'd be 8 bytes instead of 12. And sets end up having just 4 byte entries. I'm guessing that string dicts are the most common (hence the specialized lookupdict_string routine), and since strings already contain their hash, this would probably mitigate the performance impact. One could also add a hash to Tuples since they are immutable. If this isn't a totally stupid idea, I'd be happy to volunteer to try the experiment and run any suggested tests.thanks!-KiratPS any opinion on making _Py_StringEq a macro? inline function would be nice but I hesitate to bring up the C/C++ debate, both languages suck in their own special way ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 343 update (with statement context terminology)
For those not following along at home, I've now updated PEP 343 to clarify my originally intended meanings for various terms, and to record the fact that we don't currently have a consensus on python-dev that those are the right definitions. As written up in the PEP, I plan to propagate those interpretations throughout the documentation and implementation for 2.5a2, so we have at least one release using my original vision to see if the terminology actually all hangs together sensibly the way I believe it does :) Anthony, I'd also like it if we could include something specific in the release announcement asking folks to go over the relevant documentation so we have some feedback to work with on whether the updated documentation makes sense or not. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] With context, please
At 01:33 PM 4/23/2006 +1000, Nick Coghlan wrote: >Phillip J. Eby wrote: >>At 09:25 AM 4/22/2006 -0700, Aahz wrote: >>> EXPRESSION returns a value that the with statement uses to create a >>> context (a special kind of namespace). The context is used to >>> execute the BLOCK. The block might end normally, get terminated by >>> a break or return, or raise an exception. No matter which of those >>> things happens, the context contains code to clean up after the >>> block. >>> >>> The as NAME part is optional. If you include it, you can use NAME >>> in your BLOCK >>> >>>Then a bit later: >>> >>> The protocol used by the with statement is called the context >>> management protocol, and objects implementing it are context >>> managers. >As far as I can tell, Aahz's book doesn't currently say anything that >favours one interpretation over the other (which is probably a good thing >from Aahz's point of view :). Read the first sentence again: "EXPRESSION returns a value that the with statement uses to *create* a context" (emphasis added). It doesn't say that the value *is* the context, and if anything, the second excerpt supports that by implying that the context manager is the thing passed to the "with" statement. I could be wrong, Nick, but it really looks to me like you're the only person who's gotten this particular interpretation, and I at least don't understand what the supporting argument for your interpretation is, other than that it's what you always meant it to be. (Whereas there are obvious rationales for __context__ returning a "context" object, and for a context manager being longer lived than contexts.) I also haven't seen you explain your theory of why you should be able to take the object returned from __context__ and pass it into the "with" statement. Given those things, I'd suggest that the consensus is overwhelmingly in favor of the "with contextmanager" terminology, or something similar, and that should be what gets used. I don't think it's fair to say that there's a lack of consensus in the sense that it could go either way like a 50-50 or 60-40. So far, it's 5:1 against the object passed to "with" being called the "context", even if Guido abstains. Also, there haven't been any complaints of the a1 documentation being unclear on this; rather, it was the *PEP* that was considered confusing. So in effect, we already tested this in a1, with one interpretation in the docs and another in the PEP, and the documenation won. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reducing memory overhead for dictionaries by removing me_hash
Kirat Singh wrote: > Hi, this is my first python dev post, so please forgive me if this topic > has already been discussed. To my knowledge, this hasn't been discussed before. > It seemed to me that removing me_hash from a dict entry would save 2/3 > of the space used by dictionaries and also improve alignment of the > entries since they'd be 8 bytes instead of 12. And sets end up having > just 4 byte entries. How did you compute the saving of 2/3? If one field goes and two fields stay, it saves 1/3, no? Also, the "improved" alignment isn't that much of an improvement on a 32-bit system, AFAICT. Likewise, on a 64-bit system, me_hash consumes 8 bytes (regardless of whether a long is 4 or 8 bytes), so the saving would be 1/3, and the alignment would change (not improve) from 8 to 16. > I'm guessing that string dicts are the most common (hence the > specialized lookupdict_string routine), and since strings already > contain their hash, this would probably mitigate the performance impact. > One could also add a hash to Tuples since they are immutable. You implicitly give the reason for the me_hash field: performance. This is a classical space-vs-time trade-off, currently leaning towards more-space-less-time. You want the trade-off be the other way: less-space-more-time. As you might not be aware of how much more time it will take, consider these uses of the me_hash field: - this is open addressing, so when performing a lookup 1. computes the hash of the key 2. looks at the address according to the hash 3. If there is no key at that address, lookup fails (and a free slot is found) 4. if the hash of the entry at that address is equal, it compares the keys. If they are equal, lookup succeeds. 5. If the hashes are not equal, or the keys are not equal, an alternative address is computed, and search continues with step 3. Under your algorithm, you would have additional object comparisons in case of hash values that translate to the same address. This might slow down dictionaries that are close to being full. - when the dictionary is resized, all entries need to be rehashed. So if me_hash is removed, you have many more PyObject_Hash calls on resizing. This would likely slow down resizing significantly. > If this isn't a totally stupid idea, I'd be happy to volunteer to try > the experiment and run any suggested tests. I don't see much value in saving a few bytes, and I expect that performance will go down noticeably. Still, only an experiment could show what the real impact is. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
