Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Stephen J. Turnbull]

PJ Eby writes:
 > On Wed, Nov 21, 2012 at 12:00 AM, Stephen J. Turnbull 
 > wrote:

 > > What I care about is when I'm using Gorgon, and there's something
 > > "better" (or worse, "correct") to use in my application.
 > 
 > Hence my suggestion for an Obsoleted-By field, in which Gorgon would be
 > able to suggest alternatives.

My bad, my precise intention was to follow up on your idea (which,
credit where credit is due, I had *not* hit upon independently).  I
should have made that clear.

(I really shouldn't be answering English email at a Japanese-speaking
conference, my brain thinks it knows what it's doing but shirazuni 日
本文化が染み込む)

 > > It might be a good idea to have a just-like-Amazon
 > >
 > > While-This-Package-Is-Great-You-Might-Also-Consider:
 > >
 > > field.
 > 
 > Yeah, that's basically what Obsoleted-By is for.

Well, Obsoleted-By is pretty strong language for suggesting possible
alternatives.  But I suspect that few projects would really want to be
suggesting competitors' products *or* their own oldie-but-still-goodie
that they'd really like to obsolete ASAP (put an Obsoleted-By line in
every Python 2 distribution, anyone? :-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[PJ Eby]

On Wed, Nov 21, 2012 at 12:00 AM, Stephen J. Turnbull wrote:

> Daniel Holth writes:
>
>  > When I used Obsoletes, it meant "I am no longer developing this other
>  > package that is identical to this re-named package".
>
> But as a user I could care less!  The authors may care, but I don't
> care if Torqued "obsoletes" Gorgon, because in using Torqued I'm
> DTRT'ing even though I don't know it.
>
> What I care about is when I'm using Gorgon, and there's something
> "better" (or worse, "correct") to use in my application.
>

Hence my suggestion for an Obsoleted-By field, in which Gorgon would be
able to suggest alternatives.



> It might be a good idea to have a just-like-Amazon
>
> While-This-Package-Is-Great-You-Might-Also-Consider:
>
> field.
>

Yeah, that's basically what Obsoleted-By is for.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Stephen J. Turnbull]

Daniel Holth writes:

 > When I used Obsoletes, it meant "I am no longer developing this other
 > package that is identical to this re-named package".

But as a user I could care less!  The authors may care, but I don't
care if Torqued "obsoletes" Gorgon, because in using Torqued I'm
DTRT'ing even though I don't know it.

What I care about is when I'm using Gorgon, and there's something
"better" (or worse, "correct") to use in my application.

It might be a good idea to have a just-like-Amazon

While-This-Package-Is-Great-You-Might-Also-Consider:

field.

Tongue-in-cheeki-ly y'rs,
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Nick Coghlan]

On Wed, Nov 21, 2012 at 1:20 PM, Nick Coghlan  wrote:

> On Wed, Nov 21, 2012 at 1:10 PM, PJ Eby  wrote:
>
>> Conversely, if you have already installed a package that says it
>> "Obsoletes" another package, this does *not* tell you that the obsolete
>> package shouldn't still be installed!  A replacement project doesn't
>> necessarily share the same API, and may exist in a different package
>> namespace altogether.
>>
>
> Then that's a bug in the metadata of the project misusing "Obsoletes", and
> should be reported as such. If the new package is not a drop-in
> replacement, then it has no business claiming to obsolete the other package.
>
> I think one of the big reasons this kind of use is rare in the Python
> community is that project name changes are almost always accompanied by
> *package* name changes, and as soon as you change the package name, you're
> changing the public API, and thus it is no longer appropriate to use
> Provides or Obsoletes, as the renamed project is no longer a drop-in
> replacement for the original.
>

I realised that my comments above are more about the appropriate use of
"Provides", rather than "Obsoletes". For a practically useful "Obsoletes",
I think I'm inclined to agree with you, as "Obsoleted-By" provides a way
for a maintainer to explicitly declare that a project is no longer
receiving updates, and users should migrate to the replacement project if
they want to continue to receive fixes and improvements. The current
version of "Obsoletes" is, as Daniel describes, really only useful as
documentation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] operator.attrgetter(attr[, args...]) etc.

[Leo]

Sorry the python issue tracker seems broken (I cannot log in). So I am
posting it here.

In the doc:

  operator.attrgetter(attr[, args...])
  operator.itemgetter(item[, args...])
  operator.methodcaller(name[, args...])

The signatures of these functions seem confusing. ARGS is not documented
and does not match the python implementation in the doc.

Leo

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Nick Coghlan]

On Wed, Nov 21, 2012 at 1:10 PM, PJ Eby  wrote:

> Conversely, if you have already installed a package that says it
> "Obsoletes" another package, this does *not* tell you that the obsolete
> package shouldn't still be installed!  A replacement project doesn't
> necessarily share the same API, and may exist in a different package
> namespace altogether.
>

Then that's a bug in the metadata of the project misusing "Obsoletes", and
should be reported as such. If the new package is not a drop-in
replacement, then it has no business claiming to obsolete the other package.

I think one of the big reasons this kind of use is rare in the Python
community is that project name changes are almost always accompanied by
*package* name changes, and as soon as you change the package name, you're
changing the public API, and thus it is no longer appropriate to use
Provides or Obsoletes, as the renamed project is no longer a drop-in
replacement for the original.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Daniel Holth]

On Tue, Nov 20, 2012 at 7:18 PM, Jim Jewett  wrote:

> On 11/20/12, Daniel Holth  wrote:
> > On Tue, Nov 20, 2012 at 3:58 PM, Jim J. Jewett 
> > wrote:
>
> >> Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:
>
> >> > The use of multiple names in this field *must not* be used for
> >> > bundling distributions together. It is intended for use when
> >> > projects are forked and merged over time ...
>
> >> (1)  Then how *should* the "bundle-of-several-components" case be
> >> represented?
>
> > The useful way to bundle a bunch of things would be to just include them
> > all in an executable folder or zipfile with __main__.py. PEP 426 and the
> > package database would not get involved. The bundle would be distributed
> as
> > an application you can download and use, not as an sdist on PyPI.
>
> When I look at, for example, twisted, there are some fairly fine
> distinctions.
>
> I can imagine some people wanting to handle each little piece
> differently, since that is the level at which they would be replaced
> by a more efficient implementation.  That doesn't mean that someone
> using the default should have to manage 47 separate little packages
> individually.
>
> Also note that ZODB is mentioned as a bundling example in the current
> (2012-11-14) PEP.  What does the PEP recommend that they do?  Stop
> including transaction?  Keep including it but stop 'Provides-Dist'-ing
> it?
>
> The current PEP also specifies that "This field must include the
> project identified in the Name field, followed by the version : Name
> (Version)." but the examples do not always include version.  Why is
> the MUST there?
>

ZODB is a bad example. The word "bundling" will be struck from the PEP
entirely. Two sdists should not be combined into one sdist when both
packages are still being developed.

If A and B are merged into a single PyPI package C, and A and B will no
longer be developed, then C may Provides-Dist A and B.

http://www.python.org/dev/peps/pep-0426/#requires-dist-multiple-use

No MUST on the Requires-Dist version. If no version is there, it should
satisfy any version requirement.

Is there some way to distinguish between concrete and abstract
> provisions?  For example, if  MyMail (2012.11.10)  includes
> 'Provides-Dist: email', does that really get parsed as 'Provides-Dist:
> email (2012.11.10)'?
>

No.

>> (2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
>> The only difference I can see is that it may be a bit more polite
>> to people who do want to install multiple versions of a (possibly
>> abstract) package.

 > The intent of Provides and Obsoletes is different. Obsoletes would not
> > satisfy a requirement during dependency resolution.
>
> > The RPM guide explains a similar system:
>
> As best I can understand, Obsoletes means "Go ahead and uninstall that
> other package."
> Saying that *without* providing the same functionality seems like a
> sneaky spelling of "Please break whatever relies on that other
> package."
>
> I'm willing to believe that there is a more useful meaning.  I'm also
> willing to believe that they are logically redundant but express
> different intentions.  The current wording doesn't tell me which is
> true.  (Admittedly, that is arguably an upstream bug with other
> package systems, but you should still either fix it or explicitly
> delegate the definitions.)
>
> And as long as I'm asking for clarification, can foopkg-3.4 obsolete
> foopgk3.2?  If not, is it a semantics problem, or just not idiomatic?
> If so, does it have a precise meaning, such as "no longer
> interoperates with"?
>

When I used Obsoletes, it meant "I am no longer developing this other
package that is identical to this re-named package". The system of
requirements/conflicts (as the RPM system) does not appear to be entirely
orthogonal.

And now that I've looked more carefully ...
>
> Can a "Key: Value" pair be continued onto another line?  The syntax
> description under "Metadata Files" does not say so, but later text
> suggests that either leading whitespace or a leading tab specifically
> (from the example code) will work.  (And is description a special
> case?)
>

Description (now in the payload, please) is the only field that is commonly
multi-line.

Any field could continue onto the next line as far as the parser is
concerned. It probably would not make sense.

Is the payload assumed to be utf8 text?  Can it be itself a mime message?
>

The entire file needs to be utf-8. The payload is assumed to be utf-8 text
in this version. Wouldn't a mime message also be utf-8 text? (we wouldn't
know what to do with it)


> Are there any restrictions on 'Name'?  e.g., Can the name include
> spaces?  line breaks?  Must it be a valid python identifier?  A valid
> python qualname?
>

setuptools constrains it to alphanumeric characters. Metadata 1.3 doesn't
say.

'Version' says that it must be in the format specified in PEP 386.
> Unfortunately, it doesn't say which part of 386.  Do you mean t

Re: [Python-Dev] Accept just PEP-0426

[Nick Coghlan]

On Wed, Nov 21, 2012 at 12:44 PM, PJ Eby  wrote:

>
> Can anybody suggest an *actual* use case for "Obsoletes", and explain how
> it is supposed to work in software?  The last time this discussion came up,
> nobody had any use cases that stood up to the "how's that actually going to
> work and/or help?" test.  Here's a post of mine summarizing this and
> related points in the previous thread:
>
>   http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html
> The problem is that the above *makes no sense*.  "Torqued Python" and
> "Gorgon" are veiled pseudonyms for Twisted and Medusa  and Twisted is
> not actually a plug-and-play substitute for Medusa, AFAIK.
>

Sure. This is an RPM example, but exactly the same thing applies at the
Python level.

One of the dependencies of PulpDist (a directory mirroring tool I wrote),
is the Pulp project (originally just an RPM mirroring tool, but now with
plugin-based support for mirroring other things). The upstream version of
Pulp that I currently use is missing Kerberos login support, so I have
patched that in via RPMs patching features.

To avoid messing up others sharing the internal yum repo where this is
published, I actually use the Provides/Conflict/Obsoletes features of RPM
to make sure my patched and renamed copy and the upstream version don't
interfere with each other (and certainly can't be installed on the same
system, as they would trample all over each other by attempting to install
the same files).

Mostly though, these labelling tools are especially useful for internal
forks and mergers - the ones you *don't* share with the wider internet,
except perhaps in the form of upstream patches (For example:
https://bugzilla.redhat.com/show_bug.cgi?id=831937). On a public index,
drop-in replacements are *always* going to be controversial from a social
point of view, which is why there are only two current examples on PyPI I
am aware of (i.e. distribute vs setuptools and Pillow vs PIL). The first
was essentially a hostile fork, while the latter started as an attempt to
provide decent packaging support when the current maintainer didn't show
any interest in doing so. In such cases, it is absolutely essential that
the *forking* project is able to declare that it is a replacement for the
original project. It is then up to the community to decide whether or not
the claims of being a suitable replacement are valid, which will be shown
most clearly in relative uptake numbers between the original project and
the forked one.

I do consider it unfortunate that Python has only copied 3 of the 4 RPM
dependency management fields (i.e. only Provides, Requires, and Obsoletes,
without copying the more value neutral Conflicts) and I also prefer the
"capability" terminology in the Fedora RPM guide that makes it clear that
these are really arbitrary strings from a tooling point of view that only
match the package name by convention.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[PJ Eby]

On Tue, Nov 20, 2012 at 6:43 PM, Daniel Holth  wrote:

> No. We trust the packages we install, including the way they decide to use
> the metadata. A bad package could delete all our files or cause dependency
> resolution to fail. Mostly they won't.
>

That's sort of beside the point.  The *only* use case which Obsoletes
provides over Obsoleted-By is that it allows third parties to unilaterally
advertise their forked project as a substitute for the original, and maybe
block users from switching back to the un-forked project -- regardless of
the status of the original project or the consent of the original project's
maintainer.

This use case, however, benefits nobody besides the forkers.  There are
many other legitimate channels by which the forkers can advertise
themselves as a replacement for their parent project, and no reason for the
installing end user to be bothered with the subject, except in case of a
conflict.

For somebody obsoleting their own package, on the other hand, it's likely
well worth the effort to at least update their PyPI metadata to reflect the
change in status -- especially if this can be done through the web
interface.   It's likely they would wish to update their description as
well, to notify human beings of the change.

But here's the thing that kills "Obsoletes" dead in the first place as a
practical tool: unless installers use a PyPI search before installing
*every single project*, there is no way for them to realize that the
obsoleting package exists!

By contrast, if a package is "Obsoleted-By", then installing that package
(or declaring a dependency on it) provides an opportunity to inform the
user of the need to make a transition.  This can't be done with an
"Obsoletes" field.

Conversely, if you have already installed a package that says it
"Obsoletes" another package, this does *not* tell you that the obsolete
package shouldn't still be installed!  A replacement project doesn't
necessarily share the same API, and may exist in a different package
namespace altogether.

In short, "Obsoletes" is virtually *useless* as a machine-consumed metadata
field, because there is nothing you can actually do with it in a practical
installer.

I'm against adding more fields to the metadata which do not have a
specification for how they should be used in practice; the presence of such
fields has been a problem with most of the preceding metadata specs, IMO.


> I re-named a package once just because I did not like the name. I used
"Obsoletes" for that. It is documentation.

Note that "Obsoleted-By" would also serve that use case, and have the
additional benefit of being able to notify people who install new copies of
the replaced project.

(By the way Daniel, I'm sorry I didn't comment on this PEP sooner; I'd
forgotten about the previous PEP 345 rehashing in 2010, or rather, I just
sort of assumed that the results of that discussion had been incorporated
into the newer PEP, and didn't notice the reappearance of the noise fields
until your call for approval just now.  Sorry!)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Daniel Holth]

On Tue, Nov 20, 2012 at 9:44 PM, PJ Eby  wrote:

> On Tue, Nov 20, 2012 at 5:01 PM, Daniel Holth  wrote:
>
> http://www.python.org/dev/peps/pep-0314/ says:
>>
>>   The most common use of this field will be in case a package name
>>   changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
>>   When you install Torqued Python, the Gorgon package should be
>>   removed.
>>
>>   Example:
>>
>>   Obsoletes: Gorgon
>>
>>
>> They mean pretty much what the same words mean in RPM and do not need
>> further bikeshedding.
>>
>
> The problem is that the above *makes no sense*.  "Torqued Python" and
> "Gorgon" are veiled pseudonyms for Twisted and Medusa  and Twisted is
> not actually a plug-and-play substitute for Medusa, AFAIK.
>
> Can anybody suggest an *actual* use case for "Obsoletes", and explain how
> it is supposed to work in software?  The last time this discussion came up,
> nobody had any use cases that stood up to the "how's that actually going to
> work and/or help?" test.  Here's a post of mine summarizing this and
> related points in the previous thread:
>
>

Again I didn't write any of this.

Someone mentioned ZODB + transaction. The PEP should have used the word
"merged" instead of "bundled". When two packages become one, and the
redundant package is no longer being developed, Provides-Dist can be used.

I re-named a package once just because I did not like the name. I used
"Obsoletes" for that. It is documentation.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[PJ Eby]

On Tue, Nov 20, 2012 at 5:01 PM, Daniel Holth  wrote:

> http://www.python.org/dev/peps/pep-0314/ says:
>
>   The most common use of this field will be in case a package name
>   changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
>   When you install Torqued Python, the Gorgon package should be
>   removed.
>
>   Example:
>
>   Obsoletes: Gorgon
>
>
> They mean pretty much what the same words mean in RPM and do not need
> further bikeshedding.
>

The problem is that the above *makes no sense*.  "Torqued Python" and
"Gorgon" are veiled pseudonyms for Twisted and Medusa  and Twisted is
not actually a plug-and-play substitute for Medusa, AFAIK.

Can anybody suggest an *actual* use case for "Obsoletes", and explain how
it is supposed to work in software?  The last time this discussion came up,
nobody had any use cases that stood up to the "how's that actually going to
work and/or help?" test.  Here's a post of mine summarizing this and
related points in the previous thread:

  http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Jim Jewett]

On 11/20/12, Daniel Holth  wrote:
> On Tue, Nov 20, 2012 at 3:58 PM, Jim J. Jewett 
> wrote:

>> Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:

>> > The use of multiple names in this field *must not* be used for
>> > bundling distributions together. It is intended for use when
>> > projects are forked and merged over time ...

>> (1)  Then how *should* the "bundle-of-several-components" case be
>> represented?

> The useful way to bundle a bunch of things would be to just include them
> all in an executable folder or zipfile with __main__.py. PEP 426 and the
> package database would not get involved. The bundle would be distributed as
> an application you can download and use, not as an sdist on PyPI.

When I look at, for example, twisted, there are some fairly fine distinctions.

I can imagine some people wanting to handle each little piece
differently, since that is the level at which they would be replaced
by a more efficient implementation.  That doesn't mean that someone
using the default should have to manage 47 separate little packages
individually.

Also note that ZODB is mentioned as a bundling example in the current
(2012-11-14) PEP.  What does the PEP recommend that they do?  Stop
including transaction?  Keep including it but stop 'Provides-Dist'-ing
it?

The current PEP also specifies that "This field must include the
project identified in the Name field, followed by the version : Name
(Version)." but the examples do not always include version.  Why is
the MUST there?

Is there some way to distinguish between concrete and abstract
provisions?  For example, if  MyMail (2012.11.10)  includes
'Provides-Dist: email', does that really get parsed as 'Provides-Dist:
email (2012.11.10)'?



>> (2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
>> The only difference I can see is that it may be a bit more polite
>> to people who do want to install multiple versions of a (possibly
>> abstract) package.

> The intent of Provides and Obsoletes is different. Obsoletes would not
> satisfy a requirement during dependency resolution.

> The RPM guide explains a similar system:

As best I can understand, Obsoletes means "Go ahead and uninstall that
other package."
Saying that *without* providing the same functionality seems like a
sneaky spelling of "Please break whatever relies on that other
package."

I'm willing to believe that there is a more useful meaning.  I'm also
willing to believe that they are logically redundant but express
different intentions.  The current wording doesn't tell me which is
true.  (Admittedly, that is arguably an upstream bug with other
package systems, but you should still either fix it or explicitly
delegate the definitions.)

And as long as I'm asking for clarification, can foopkg-3.4 obsolete
foopgk3.2?  If not, is it a semantics problem, or just not idiomatic?
If so, does it have a precise meaning, such as "no longer
interoperates with"?

And now that I've looked more carefully ...

Can a "Key: Value" pair be continued onto another line?  The syntax
description under "Metadata Files" does not say so, but later text
suggests that either leading whitespace or a leading tab specifically
(from the example code) will work.  (And is description a special
case?)

Is the payload assumed to be utf8 text?  Can it be itself a mime message?

Are there any restrictions on 'Name'?  e.g., Can the name include
spaces?  line breaks?  Must it be a valid python identifier?  A valid
python qualname?

'Version' says that it must be in the format specified in PEP 386.
Unfortunately, it doesn't say which part of 386.  Do you mean that it
must be acceptable to verlib.NormalizedVersion without first having to
call suggest_normalized_version?

'Summary' specifies that it must be one line.  Is there a character
limit, or do you just mean "no line breaks"?  Do you want to add a
"Should be less than 80 characters" or some such, based on typical
tool presentation?  Would it be worth repeating the advice that longer
descriptions should go in the payload, after all headers?  (Otherwise,
they have to find 'Description' *and* notice that it is deprecated and
figure out what to do instead.)

Under 'Description', it isn't entirely clear whether what terminates
the field.  "Multiple paragraphs" suggests that there can be multiple
lines, but I'm guessing that -- in practice -- they have to be a
single logical line, with all but the first starting with whitespace.

Under 'Classifier', is PEP 301 really the current authority for
classifiers?  I would prefer at least a reference to
http://pypi.python.org/pypi?%3Aaction=list_classifiers demonstrating
which classifiers are currently meaningful.

Under 'Requires-Dist', there is an unclosed parenthesis.

Does the 'Setup-Requires-Dist' set implicitly include the
'Requires-Dist' set, or should a package be listed both ways if it is
required at both setup and runtime?

The Summary of Differences from PEP 345 mentions changes to
Requires-Dis

Re: [Python-Dev] Accept just PEP-0426

[Glenn Linderman]

On 11/20/2012 1:48 PM, PJ Eby wrote:
(We've been over this before, the last time this discussion came up on 
the Distutils-SIG for a previous Metadata PEP a year or two back, but 
here goes)


Thanks. I wasn't over there. Makes it clear that clarifying PEPs to 
reflect discussions is a good idea.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Toshio Kuratomi]

On Tue, Nov 20, 2012 at 06:43:32PM -0500, Daniel Holth wrote:
> No. We trust the packages we install, including the way they decide to use
> the metadata. A bad package could delete all our files or cause dependency
> resolution to fail. Mostly they won't.
> 
Agreed.  And this is closer to the way that distributions' tools have to
operate than they'd want to :-(  Within the distribution we like to pretend
that we only need to care about the packages that we generate.  But we also
know that whether or not we support it, ordinary users will install pacakges
from outside of our walls.  That means that the packaging tools that we
create will need to deal with things that we might not condone within our
"presumed authority".

We trust that people are going to do more or less the right thing with
the tools we offer.  Once in a while they don't but by and large they do.

-Toshio

> Daniel Holth
> 
> On Nov 20, 2012, at 5:26 PM, Vinay Sajip  wrote:
> 
> > Daniel Holth  gmail.com> writes:
> > 
> >> They mean pretty much what the same words mean in RPM and do not need 
> >> further
> >> bikeshedding.
> > 
> > But isn't it the case that the scenarios are different because in the case 
> > of
> > RPMs, we have a presumed authority which can determine e.g. what obsoletes 
> > what,
> > whereas with Python distributions, there's no central authority that has 
> > this
> > function?
> > 
> > Regards,
> > 
> > Vinay Sajip
> > 
> > 
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/a.badger%40gmail.com


pgpvh6CUSBynq.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Daniel Holth]

No. We trust the packages we install, including the way they decide to use the 
metadata. A bad package could delete all our files or cause dependency 
resolution to fail. Mostly they won't.

Daniel Holth

On Nov 20, 2012, at 5:26 PM, Vinay Sajip  wrote:

> Daniel Holth  gmail.com> writes:
> 
>> They mean pretty much what the same words mean in RPM and do not need further
>> bikeshedding.
> 
> But isn't it the case that the scenarios are different because in the case of
> RPMs, we have a presumed authority which can determine e.g. what obsoletes 
> what,
> whereas with Python distributions, there's no central authority that has this
> function?
> 
> Regards,
> 
> Vinay Sajip
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514: Fix regression causing a traceback when sys.path[0] is None

[Barry Warsaw]

On Nov 20, 2012, at 05:35 PM, Terry Reedy wrote:

>On 11/20/2012 3:35 PM, barry.warsaw wrote:
>
>>   for entry in path:
>> +if not isinstance(entry, (str, bytes)):
>> +continue
>
>Given that a non-(str,bytes) entry could indicate a programming error, should
>a warning be emitted before continuing?

That's not what happens in Python 3.2.  In fact, this bug report was triggered
by someone who was already inserting None to sys.path[0].  It is silently
ignored in Python 3.2, but tracebacked in 3.3.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514: Fix regression causing a traceback when sys.path[0] is None

[Barry Warsaw]

On Nov 20, 2012, at 05:12 PM, Brett Cannon wrote:

>Should you also insert None into sys.path_importer_cache to signify there
>is no finder for the path entry? I guess the real problem with that is
>there is no guarantee the path entry is hashable, so that probably won't
>work. So nevermind. =)

I explicitly popped None from path_importer_cache, to try to force it to
ignore the cache.  Maybe it's not totally correct (and in practice probably
doesn't matter), but it WFM.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Vinay Sajip]

Daniel Holth  gmail.com> writes:

> They mean pretty much what the same words mean in RPM and do not need further
> bikeshedding.

But isn't it the case that the scenarios are different because in the case of
RPMs, we have a presumed authority which can determine e.g. what obsoletes what,
whereas with Python distributions, there's no central authority that has this
function?

Regards,

Vinay Sajip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514: Fix regression causing a traceback when sys.path[0] is None

[Brett Cannon]

Should you also insert None into sys.path_importer_cache to signify there
is no finder for the path entry? I guess the real problem with that is
there is no guarantee the path entry is hashable, so that probably won't
work. So nevermind. =)


On Tue, Nov 20, 2012 at 3:35 PM, barry.warsaw wrote:

> http://hg.python.org/cpython/rev/291406748217
> changeset:   80529:291406748217
> branch:  3.3
> parent:  80527:4537dd27b2dc
> user:Barry Warsaw 
> date:Tue Nov 20 15:22:51 2012 -0500
> summary:
>   - Issue #16514: Fix regression causing a traceback when sys.path[0] is
> None
>   (actually, any non-string or non-bytes type).
>
> files:
>   Doc/library/sys.rst  |4 +-
>   Doc/reference/import.rst |   24 +-
>   Lib/importlib/_bootstrap.py  |2 +
>   Lib/test/test_importlib/import_/test_path.py |   25 +-
>   Misc/NEWS|3 +
>   Python/importlib.h   |  130 +
>   6 files changed, 111 insertions(+), 77 deletions(-)
>
>
> diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
> --- a/Doc/library/sys.rst
> +++ b/Doc/library/sys.rst
> @@ -783,7 +783,9 @@
> current directory first.  Notice that the script directory is inserted
> *before*
> the entries inserted as a result of :envvar:`PYTHONPATH`.
>
> -   A program is free to modify this list for its own purposes.
> +   A program is free to modify this list for its own purposes.  Only
> strings
> +   and bytes should be added to :data:`sys.path`; all other data types are
> +   ignored during import.
>
>
> .. seealso::
> diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst
> --- a/Doc/reference/import.rst
> +++ b/Doc/reference/import.rst
> @@ -540,7 +540,10 @@
>  implementation-specific defaults.  Entries in :data:`sys.path` can name
>  directories on the file system, zip files, and potentially other
> "locations"
>  (see the :mod:`site` module) that should be searched for modules, such as
> -URLs, or database queries.
> +URLs, or database queries.  Only strings and bytes should be present on
> +:data:`sys.path`; all other data types are ignored.  The encoding of bytes
> +entries is determined by the individual :term:`path entry finders  entry
> +finder>`.
>
>  The :term:`path based finder` is a :term:`meta path finder`, so the import
>  machinery begins the :term:`import path` search by calling the path
> @@ -563,14 +566,17 @@
>  the path based finder to perform the path entry search again [#fnpic]_.
>
>  If the path entry is not present in the cache, the path based finder
> iterates
> -over every callable in :data:`sys.path_hooks`.  Each of the
> -:term:`path entry hooks ` in this list is called with a
> -single argument, the path entry to be searched.  This callable may either
> -return a :term:`path entry finder` that can handle the path entry, or it
> may
> -raise :exc:`ImportError`.
> -An :exc:`ImportError` is used by the path based finder to signal that the
> hook
> -cannot find a :term:`path entry finder` for that :term:`path entry`.  The
> -exception is ignored and :term:`import path` iteration continues.
> +over every callable in :data:`sys.path_hooks`.  Each of the :term:`path
> entry
> +hooks ` in this list is called with a single argument,
> the
> +path entry to be searched.  This callable may either return a :term:`path
> +entry finder` that can handle the path entry, or it may raise
> +:exc:`ImportError`.  An :exc:`ImportError` is used by the path based
> finder to
> +signal that the hook cannot find a :term:`path entry finder` for that
> +:term:`path entry`.  The exception is ignored and :term:`import path`
> +iteration continues.  The hook should expect either a string or bytes
> object;
> +the encoding of bytes objects is up to the hook (e.g. it may be a file
> system
> +encoding, UTF-8, or something else), and if the hook cannot decode the
> +argument, it should raise :exc:`ImportError`.
>
>  If :data:`sys.path_hooks` iteration ends with no :term:`path entry finder`
>  being returned, then the path based finder's :meth:`find_module()` method
> diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
> --- a/Lib/importlib/_bootstrap.py
> +++ b/Lib/importlib/_bootstrap.py
> @@ -1281,6 +1281,8 @@
>  #  the list of paths that will become its __path__
>  namespace_path = []
>  for entry in path:
> +if not isinstance(entry, (str, bytes)):
> +continue
>  finder = cls._path_importer_cache(entry)
>  if finder is not None:
>  if hasattr(finder, 'find_loader'):
> diff --git a/Lib/test/test_importlib/import_/test_path.py
> b/Lib/test/test_importlib/import_/test_path.py
> --- a/Lib/test/test_importlib/import_/test_path.py
> +++ b/Lib/test/test_importlib/import_/test_path.py
> @@ -1,15 +1,14 @@
>  from importlib import _bootstrap
>  from importlib import machinery
> +from importlib imp

Re: [Python-Dev] Accept just PEP-0426

[Daniel Holth]

I think the Metadata 1.1 treatment of these concepts is in some ways
better. (Metadata 1.2 added the -Dist suffix to the fields in an attempt to
make it clear that dependency names are PyPI names and not "import x"
names.)

http://www.python.org/dev/peps/pep-0314/ says:

Provides (multiple use)

  Each entry contains a string describing a package or module that
  will be provided by this package once it is installed.  These
  strings should match the ones used in Requirements fields.  A
  version declaration may be supplied (without a comparison
  operator); the package's version number will be implied if none
  is specified.

  Example:

  Provides: xml
  Provides: xml.utils
  Provides: xml.utils.iso8601
  Provides: xml.dom
  Provides: xmltools (1.3)

Obsoletes (multiple use)

  Each entry contains a string describing a package or module
  that this package renders obsolete, meaning that the two packages
  should not be installed at the same time.  Version declarations
  can be supplied.

  The most common use of this field will be in case a package name
  changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
  When you install Torqued Python, the Gorgon package should be
  removed.

  Example:

  Obsoletes: Gorgon


They mean pretty much what the same words mean in RPM and do not need
further bikeshedding.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Donald Stufft]

On Tuesday, November 20, 2012 at 4:48 PM, PJ Eby wrote:
> Words
I agree that obsoletes is terrible, it's very specific and not something we 
particularly require. I'd much rather just have a generic "conflicts". ___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[PJ Eby]

On Tue, Nov 20, 2012 at 4:07 PM, Glenn Linderman wrote:

>  On 11/20/2012 12:46 PM, PJ Eby wrote:
>
>  I personally don't think that forks claiming to "provide" something is
> really a good thing to encourage; ISTM that saying a package *conflicts*
> with another is more accurate, e.g. distribute Conflicts-Dist setuptools.
> I also think distributions should say they are obsoleted, rather than
> allowing other distributions to obsolete them.
>
>  Obsolete distributions won't say they are obsoleted, unless they receive
> further maintenance. However, if the distribution is obsolete because the
> maintainer has lost interest, they won't receive further maintenance.
>

(We've been over this before, the last time this discussion came up on the
Distutils-SIG for a previous Metadata PEP a year or two back, but here
goes)

Obsoleting a package is for handling renames and support transitions.  For
example, if it actually did anything to do so, I'd mark RuleDispatch as
obsoleted-by PEAK, the Pylons folks might mark some version of that as
obsoleted-by Pyramid, etc.  To put it another way, marking a package
obsolete is part of deprecation and replacement, not an unsubstantiated
third-party claim about the maintenance status of an unrelated project.

If a package is *actually* dead, there's no real point to declaring that
something else obsoletes it, and certainly no reason to put it in metadata
form.  Otherwise, we could have Twisted claiming to obsolete GEvent and
vice-versa at the same time.  Which one should an installer believe?  It
makes no sense in a standard where the project's maintainers can say
whatever they want about somebody else's project.  The scope of authority
for automatically-consumed metadata should *only* encompass the project
that provided the metadata.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Daniel Holth]

On Tue, Nov 20, 2012 at 3:58 PM, Jim J. Jewett  wrote:

>
>
> Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:
>
> > The use of multiple names in this field *must not* be used for
> > bundling distributions together. It is intended for use when
> > projects are forked and merged over time ...
>
> (1)  Then how *should* the "bundle-of-several-components" case be
> represented?
>
> (2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
> The only difference I can see is that it may be a bit more polite
> to people who do want to install multiple versions of a (possibly
> abstract) package.
>

The useful way to bundle a bunch of things would be to just include them
all in an executable folder or zipfile with __main__.py. PEP 426 and the
package database would not get involved. The bundle would be distributed as
an application you can download and use, not as an sdist on PyPI.

The intent of Provides and Obsoletes is different. Obsoletes would not
satisfy a requirement during dependency resolution.

The RPM guide explains a similar system:
This brings the total to four types of dependencies that the RPM system
tracks:

   - Requires, which tracks the capabilities a package requires
   - Provides, which tracks the capabilities a package provides for other
   packages
   - Conflicts, which describes the capabilities that if installed,
   conflict with capabilities in a package
   - Obsoletes, which describes the capabilities that this package will
   make obsolete

Packages advertise this dependency information. Each dependency holds the
type, such as requires, a capability, such as a shared library or a package
name, and optionally a version number, such as requiring the python package
at a version number greater than or equal to 2.2 (python >= 2.2).

http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-dependencies.html#RPM_Guide-Dependencies-obsoletes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Glenn Linderman]

On 11/20/2012 12:46 PM, PJ Eby wrote:
On Tue, Nov 20, 2012 at 11:49 AM, Vinay Sajip > wrote:


Also: what happens when a requirement is for setuptools (>= X.Y),
but the
distribute fork hasn't kept pace, and so only supports setuptools
at a lower
version than X.Y? I take it we're entirely comfortable with installing
setuptools X.Y in that case? How would you ensure the right
setuptools is
always loaded, since presumably both are on sys.path?


Egg-based tools don't have any problem with this, since they set 
sys.path to include the eggs needed for the running program.  Other 
tools will have to tell the user and let them work it out, e.g. by 
using a different virtualenv.


I personally don't think that forks claiming to "provide" something is 
really a good thing to encourage; ISTM that saying a package 
*conflicts* with another is more accurate, e.g. distribute 
Conflicts-Dist setuptools.  I also think distributions should say they 
are obsoleted, rather than allowing other distributions to obsolete them.


Obsolete distributions won't say they are obsoleted, unless they receive 
further maintenance. However, if the distribution is obsolete because 
the maintainer has lost interest, they won't receive further maintenance.



That is, centralized packaging systems rely on a central authority to 
resolve issues of who provides what and obsoletes what; there's an 
implicit "x obsoletes y [by decree of semi-independent third-party z]".


However, in Python package metadata, it's "x obsoletes y [by decree of 
x]".  IMO, this should be reversed to, "Y is obsoleted by x [by decree 
of y]", and "installing Y will conflict with X [by decree of X]", so 
that in each case the scope of authority for the statement is clear.


That is, in each case (conflict or obsolescence), the project's 
developers are declaring under what conditions they will not be 
supporting an installation.  In the case of obsolescence, the 
developer is saying, "this is being phased out, you should use that 
other thing instead".  In the case of forks, the developer is saying, 
"If you install both versions, something's gonna break."


Note that installation conflict is a more conservative claim anyway: a 
conflict between forked "foobar" packages is permanent, in the sense 
that it doesn't matter what versions of both packages you're 
interested in: they both want to install a foobar/__init__.py.  (Of 
course, installers can and should detect that condition automatically, 
but not until they download the package first.)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Jim J. Jewett]

 
Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:

> The use of multiple names in this field *must not* be used for
> bundling distributions together. It is intended for use when
> projects are forked and merged over time ...

(1)  Then how *should* the "bundle-of-several-components" case be
represented?

(2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
The only difference I can see is that it may be a bit more polite
to people who do want to install multiple versions of a (possibly
abstract) package.


-jJ

-- 

If there are still threading problems with my replies, please 
email me with details, so that I can try to resolve them.  -jJ

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[PJ Eby]

On Tue, Nov 20, 2012 at 11:49 AM, Vinay Sajip wrote:

> Also: what happens when a requirement is for setuptools (>= X.Y), but the
> distribute fork hasn't kept pace, and so only supports setuptools at a
> lower
> version than X.Y? I take it we're entirely comfortable with installing
> setuptools X.Y in that case? How would you ensure the right setuptools is
> always loaded, since presumably both are on sys.path?
>

Egg-based tools don't have any problem with this, since they set sys.path
to include the eggs needed for the running program.  Other tools will have
to tell the user and let them work it out, e.g. by using a different
virtualenv.

I personally don't think that forks claiming to "provide" something is
really a good thing to encourage; ISTM that saying a package *conflicts*
with another is more accurate, e.g. distribute Conflicts-Dist setuptools.
I also think distributions should say they are obsoleted, rather than
allowing other distributions to obsolete them.

That is, centralized packaging systems rely on a central authority to
resolve issues of who provides what and obsoletes what; there's an implicit
"x obsoletes y [by decree of semi-independent third-party z]".

However, in Python package metadata, it's "x obsoletes y [by decree of
x]".  IMO, this should be reversed to, "Y is obsoleted by x [by decree of
y]", and "installing Y will conflict with X [by decree of X]", so that in
each case the scope of authority for the statement is clear.

That is, in each case (conflict or obsolescence), the project's developers
are declaring under what conditions they will not be supporting an
installation.  In the case of obsolescence, the developer is saying, "this
is being phased out, you should use that other thing instead".  In the case
of forks, the developer is saying, "If you install both versions,
something's gonna break."

Note that installation conflict is a more conservative claim anyway: a
conflict between forked "foobar" packages is permanent, in the sense that
it doesn't matter what versions of both packages you're interested in: they
both want to install a foobar/__init__.py.  (Of course, installers can and
should detect that condition automatically, but not until they download the
package first.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Vinay Sajip]

Daniel Holth  gmail.com> writes:

 
> Edit the following text:

Okay, here is a possible version:
-
Provides-Dist (multiple use)

Each entry contains a string naming a requirement that is satisfied by
installing this distribution.  The entry must consist of a name and version.

This name of the project identified in the ``Name`` field is implicitly
considered as provided, with the version specified in the ``Version`` field.

The use of multiple names in this field *must not* be used for bundling
distributions together. It is intended for use when projects are forked and
merged over time, while providing essentially the same function. Multiple
names reflect the evolution of the project over time and not the bringing
together of different packages, already distributed elsewhere, in a bundle.

Thus, the 'distribute' distribution, a fork of setuptools, could say that it
``Provides-Dist`` a particular version of setuptools, to prevent setuptools
from being downloaded and installed when distribute is already installed.

If, over time, distribute evolved into a new package called 'distribute2' (for
argument's sake), then that could say that it ``Provides-Dist`` a specific
version of distribute and a specific version of setuptools.
---

Some comments on the above: I'm not entirely comfortable with a Provides-Dist
entry which does not specify a version, since it does not allow to you to test
that a requirement is actually met. So, I've removed the "optional"
qualification from the version.

Also: what happens when a requirement is for setuptools (>= X.Y), but the
distribute fork hasn't kept pace, and so only supports setuptools at a lower
version than X.Y? I take it we're entirely comfortable with installing
setuptools X.Y in that case? How would you ensure the right setuptools is
always loaded, since presumably both are on sys.path?

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Nick Coghlan]

On Wed, Nov 21, 2012 at 2:04 AM, Vinay Sajip wrote:

> Nick Coghlan  gmail.com> writes:
>
> > Provides/Requires/Obsoletes are *not* for bundling. Publishing bundled
> packages
> > on the index is bad, and people shouldn't do it.
> [detail snipped]
> > It's likely fine if an installer doesn't use sophisticated graph
> > analysis to find the "best" way to satisfy a set of requirements - you
> can
> > just as easily use it in the simple way Daniel describes of only using
> these
> > fields to check for existing locally installed packages with the
> necessary
> > capabilities, before going out to get whatever is missing from the
> package
> > index based purely on the distribution names.
>
> In which case, it seems sensible to put these constraints into the PEP,
> so that both PEP implementers and users of those implementations have these
> guidelines clarified.
>

Yes, I thought Daniel's rewording looked pretty reasonable on that front.
However, the details of how an installer uses this information is really up
to the installer developers and what their users expect/demand. It
certainly isn't *practical* to do a full dependency analysis when PyPI
doesn't provide the same kind of precalculated metadata that a yum repo
does, but that's not something that should be spelled out in the
distribution metadata PEP, any more than it is spelled out in the RPM
format spec.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Daniel Holth]

Edit the following text:


Provides-Dist (multiple use)

Each entry contains a string naming a requirement that is satisfied by
installing this distribution.  This field *must* include the project
identified in the ``Name`` field, optionally followed by the version Name
(Version).

A distribution may provide additional names, e.g. to indicate that
multiple projects have been merged into a single distribution or to
indicate that this project is a substitute for another.  For instance
distribute (a fork of setuptools) could ``Provides-Dist`` setuptools to
prevent the conflicting package from being downloaded and installed when
distribute is already installed.  A distribution that has been merged
with another might ``Provides-Dist`` the obsolete name(s) to satisfy
any projects that require the obsolete distribution's name.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

[Daniel Holth]

On Tue, Nov 20, 2012 at 9:44 AM, Nick Coghlan  wrote:

> On Tue, Nov 20, 2012 at 11:45 PM, Christian Tismer 
> wrote:
>
>>  On 20.11.12 12:39, Nick Coghlan wrote:
>>
>> On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson <
>> krist...@ccpgames.com> wrote:
>>
>>>  I’m intrigued.  I thought this was merely so that one could do
>>>
>>> python –m mypackage.mysubpackage
>>>
>>> Can you refer me to the rationale and discussion about this feature?
>>>
>>
>> It was part of a fairly long progression in changes to the command line
>> execution support :)
>>
>> Top level package execution with -m came first in 2.4, arbitrary package
>> execution for -m arrived in 2.5 (along with the runpy module), directory
>> and zipfile execution (by supplying a valid sys.path entry as the "script"
>> command line argument) was added in 2.6/3.0, and finally officially
>> supported package execution via -m only arrived in 2.7 and 3.1 (a broken
>> version of the last accidentally existed in 2.5, but that was just a
>> mistake that arose when merging the import emulations in runpy and pkgutil,
>> and had a side effect that violated at least one of the import system
>> invariants).
>>
>> In the specific case of directory and zipfile execution, discussion
>> happened on the tracker: http://bugs.python.org/issue1739468
>>
>> It was never brought up on python-dev because Guido was participating
>> directly in the tracker discussion. Unfortunately, we then also forgot to
>> mention it in the original version of the 2.6 What's New doc, so the vast
>> majority of people missed the addition :(
>>
>>
>> Hi Nick,
>>
>> thank you very much for this story and the link to the issue tracker!
>> A very good move for Python which is really not mentioned enough
>> to make people more aware of a great feature.
>>
>> I think part of this discussion should get a more prominent place
>> for gems that can be found without knowing what to search. ;-)
>>
>
> Technically, that's the "using" guide:
> http://docs.python.org/2/using/cmdline.html#interface-options (see the
> explanation of what's executable under the "

Re: [Python-Dev] Accept just PEP-0426

[Vinay Sajip]

Nick Coghlan  gmail.com> writes:

> Provides/Requires/Obsoletes are *not* for bundling. Publishing bundled 
> packages
> on the index is bad, and people shouldn't do it.
[detail snipped]
> It's likely fine if an installer doesn't use sophisticated graph
> analysis to find the "best" way to satisfy a set of requirements - you can
> just as easily use it in the simple way Daniel describes of only using these
> fields to check for existing locally installed packages with the necessary
> capabilities, before going out to get whatever is missing from the package
> index based purely on the distribution names.

In which case, it seems sensible to put these constraints into the PEP,
so that both PEP implementers and users of those implementations have these
guidelines clarified.

Regards,

Vinay Sajip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Nick Coghlan]

On Wed, Nov 21, 2012 at 1:16 AM, Vinay Sajip wrote:

> Daniel Holth  gmail.com> writes:
>
> > If you don't have Provides-Dist, then distribute must continue to bundle
> an
> > extra .egg-info directory to emulate the feature. This is more than
> enough
> > justification for me. Name: is essentially an alias for Provides-Dist:
> (or
> > vice-versa) so there is no such thing as a single-valued Provides-Dist.
> Having
> > two names for a package is just as complicated as having twenty.
>
> I'm not so sure. In the case of two names, it could be assumed that one
> was a
> fork of the other (as in the specific cases of distribute/setuptools, or
> PIL/
> Pillow). You cannot reasonably make this assumption if you have twenty
> entries
> in your Provides-Dist.
>
> > You should not implement Provides-Dist by searching for every
> Provides-Dist:
> > name on PyPI.
>
> I wasn't seriously suggesting that this approach be taken - merely
> pointing out
> that Provides-Dist isn't of much use in a metadata index.
>
> > should only use it when deciding whether to download setuptools when
> distribute
> > is already installed and a package depends on setuptools.The bundling
> term was
> > bad wording on the part of the PEP. No one should ever include
> non-renamed
> > copies of other dists in their dists "import six" vs. "import
> django.util.six".
> > I've suggested a new wording in this thread.
>
> So apart from the setuptools/distribute and PIL/Pillow scenarios, what are
> the
> scenarios where you would have 3 or more values in Provides-Dist? If they
> are
> e.g. a bundled SQLAlchemy, why would that be preferable to an entry in
> Requires-Dist?
>

Provides/Requires/Obsoletes are *not* for bundling. Publishing bundled
packages on the index is bad, and people shouldn't do it. What they're for
is tracking name changes over time, so that you can fork and rename and
merge projects without breaking the world for people that depend on your
projects (one example used in the Fedora RPM docs is the apache package
being renamed to httpd:
https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html-single/RPM_Guide/index.html#ch-dependencies
).

The fact distribute can provide setuptools and Pillow can provide PIL are
examples of the simple fork/rename case - they're designed to be drop in
replacements for the projects they forked, so it's appropriate for them to
advertise that fact in a way the deployment tools can understand. The
multi-value support is then needed if you have multiple name changes over
time (e.g. if someone were to create a distribute2 that provided both
distribute and setuptools), or if you merge two projects together (e.g. if
a popular extension to a project was folded into the main distribution for
that project).

It's likely fine if an installer doesn't use sophisticated graph analysis
to find the "best" way to satisfy a set of requirements - you can just as
easily use it in the simple way Daniel describes of only using these fields
to check for existing locally installed packages with the necessary
capabilities, before going out to get whatever is missing from the package
index based purely on the distribution names.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Vinay Sajip]

Daniel Holth  gmail.com> writes:

> If you don't have Provides-Dist, then distribute must continue to bundle an
> extra .egg-info directory to emulate the feature. This is more than enough
> justification for me. Name: is essentially an alias for Provides-Dist: (or
> vice-versa) so there is no such thing as a single-valued Provides-Dist. Having
> two names for a package is just as complicated as having twenty.

I'm not so sure. In the case of two names, it could be assumed that one was a
fork of the other (as in the specific cases of distribute/setuptools, or PIL/
Pillow). You cannot reasonably make this assumption if you have twenty entries
in your Provides-Dist.

> You should not implement Provides-Dist by searching for every Provides-Dist:
> name on PyPI.

I wasn't seriously suggesting that this approach be taken - merely pointing out
that Provides-Dist isn't of much use in a metadata index.

> should only use it when deciding whether to download setuptools when 
> distribute
> is already installed and a package depends on setuptools.The bundling term was
> bad wording on the part of the PEP. No one should ever include non-renamed
> copies of other dists in their dists "import six" vs. "import 
> django.util.six".
> I've suggested a new wording in this thread.

So apart from the setuptools/distribute and PIL/Pillow scenarios, what are the
scenarios where you would have 3 or more values in Provides-Dist? If they are
e.g. a bundled SQLAlchemy, why would that be preferable to an entry in
Requires-Dist?

Regards,

Vinay Sajip





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accept just PEP-0426

[Daniel Holth]

On Tue, Nov 20, 2012 at 9:35 AM, Vinay Sajip wrote:

> Daniel Holth  gmail.com> writes:
>
> > Mostly it seems a bit silly to have so much conversations about parts of
> the
> > pep that remain unchanged from previously accepted versions...
>
> I don't agree with the suggestion that we shouldn't discuss it because it
> was
> accepted in a previous version. Perhaps it didn't receive the right
> scrutiny at
> that time, but since it hasn't been implemented, it's reasonable to
> discuss it.
>
> ISTM that implementing it as suggested in the PEP can lead to certain
> problems,
> since it is a multi-valued field. If it is left in, then something should
> be
> said in the PEP about the potential difficulties and if/how they can be
> resolved.
>
> The difficulties I am talking about relate to dependency resolution. Given
> the
> current definition of Provides-Dist, it is possible for a package A on
> PyPI to
> "Provide" all of e.g. "A (1.0)", "B (1.2)" and "C (1.5)", and it is also
> possible for packages B and C on PyPI to provide the same (or slightly
> different) versions of logical packages of A, B, and C. This will likely
> lead
> to the need for a sophisticated dependency resolver because the dependency
> graph can get quite convoluted. (Remember, we might need to do this
> resolution
> when removing packages as well as when installing them.) I know there are
> SAT
> solvers and such, but I'm not sure we need that level of sophistication, or
> whether its complexity cost is outweighed by any benefit. Remember, we are
> managing fine without multi-valued Provides-Dist, and while a case has been
> made for virtual packages and forks (which just require a single-valued
> field),
> no compelling case has been made for bundling packages in general (I
> understand
> that such requirements might sometimes arise in certain corporate
> environments,
> but they don't seem to be a mainstream use case). Hence, no strong case has
> been made for a multi-valued "Provides" field.
>
> If we have a good index and packaging infrastructure, there is no general
> need
> for packages to bundle other packages, unless those bundled packages are
> changed
> in some way to suit the bundler's needs. In that case, I don't know how you
> could be sure that a bundled "A (1.0)" hasn't diverged from the equivalent
> package on PyPI.
>
> The "Provides" seems essentially useless in a metadata index, since if,
> when
> asked to install D which has a dependency on A, you would download and
> install
> A to resolve it rather than B or C, and I can't see when you would want to
> query the index to say "who provides A?" and then use some heuristic to
> pick
> e.g. B or C, rather than A.
>
> distlib currently contains support for the multi-valued "Provides", but I'm
> not confident that will work as expected given pathological cases like the
> example I suggested, without getting "complicated" in the Zen of Python
> sense.
> I'm not convinced that the maintenance burden of a complicated solution is
> worth the heretofore unnecessary ability to bundle stuff in arbitrary ways.
>

If you don't have Provides-Dist, then distribute must continue to bundle an
extra .egg-info directory to emulate the feature. This is more than enough
justification for me. Name: is essentially an alias for Provides-Dist: (or
vice-versa) so there is no such thing as a single-valued Provides-Dist.
Having two names for a package is just as complicated as having twenty.

You should not implement Provides-Dist by searching for every
Provides-Dist: name on PyPI. You should only use it when deciding whether
to download setuptools when distribute is already installed and a package
depends on setuptools.

The bundling term was bad wording on the part of the PEP. No one should
ever include non-renamed copies of other dists in their dists "import six"
vs. "import django.util.six". I've suggested a new wording in this thread.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

[Nick Coghlan]

On Tue, Nov 20, 2012 at 11:45 PM, Christian Tismer wrote:

>  On 20.11.12 12:39, Nick Coghlan wrote:
>
> On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson <
> krist...@ccpgames.com> wrote:
>
>>  I’m intrigued.  I thought this was merely so that one could do
>>
>> python –m mypackage.mysubpackage
>>
>> Can you refer me to the rationale and discussion about this feature?
>>
>
> It was part of a fairly long progression in changes to the command line
> execution support :)
>
> Top level package execution with -m came first in 2.4, arbitrary package
> execution for -m arrived in 2.5 (along with the runpy module), directory
> and zipfile execution (by supplying a valid sys.path entry as the "script"
> command line argument) was added in 2.6/3.0, and finally officially
> supported package execution via -m only arrived in 2.7 and 3.1 (a broken
> version of the last accidentally existed in 2.5, but that was just a
> mistake that arose when merging the import emulations in runpy and pkgutil,
> and had a side effect that violated at least one of the import system
> invariants).
>
> In the specific case of directory and zipfile execution, discussion
> happened on the tracker: http://bugs.python.org/issue1739468
>
> It was never brought up on python-dev because Guido was participating
> directly in the tracker discussion. Unfortunately, we then also forgot to
> mention it in the original version of the 2.6 What's New doc, so the vast
> majority of people missed the addition :(
>
>
> Hi Nick,
>
> thank you very much for this story and the link to the issue tracker!
> A very good move for Python which is really not mentioned enough
> to make people more aware of a great feature.
>
> I think part of this discussion should get a more prominent place
> for gems that can be found without knowing what to search. ;-)
>

Technically, that's the "using" guide:
http://docs.python.org/2/using/cmdline.html#interface-options (see the
explanation of what's executable under the "

Re: [Python-Dev] Accept just PEP-0426

[Vinay Sajip]

Daniel Holth  gmail.com> writes:

> Mostly it seems a bit silly to have so much conversations about parts of the
> pep that remain unchanged from previously accepted versions...

I don't agree with the suggestion that we shouldn't discuss it because it was
accepted in a previous version. Perhaps it didn't receive the right scrutiny at
that time, but since it hasn't been implemented, it's reasonable to discuss it.

ISTM that implementing it as suggested in the PEP can lead to certain problems,
since it is a multi-valued field. If it is left in, then something should be
said in the PEP about the potential difficulties and if/how they can be
resolved.

The difficulties I am talking about relate to dependency resolution. Given the
current definition of Provides-Dist, it is possible for a package A on PyPI to
"Provide" all of e.g. "A (1.0)", "B (1.2)" and "C (1.5)", and it is also
possible for packages B and C on PyPI to provide the same (or slightly
different) versions of logical packages of A, B, and C. This will likely lead
to the need for a sophisticated dependency resolver because the dependency
graph can get quite convoluted. (Remember, we might need to do this resolution
when removing packages as well as when installing them.) I know there are SAT
solvers and such, but I'm not sure we need that level of sophistication, or
whether its complexity cost is outweighed by any benefit. Remember, we are
managing fine without multi-valued Provides-Dist, and while a case has been
made for virtual packages and forks (which just require a single-valued field),
no compelling case has been made for bundling packages in general (I understand
that such requirements might sometimes arise in certain corporate environments,
but they don't seem to be a mainstream use case). Hence, no strong case has
been made for a multi-valued "Provides" field.

If we have a good index and packaging infrastructure, there is no general need
for packages to bundle other packages, unless those bundled packages are changed
in some way to suit the bundler's needs. In that case, I don't know how you
could be sure that a bundled "A (1.0)" hasn't diverged from the equivalent
package on PyPI.

The "Provides" seems essentially useless in a metadata index, since if, when
asked to install D which has a dependency on A, you would download and install
A to resolve it rather than B or C, and I can't see when you would want to
query the index to say "who provides A?" and then use some heuristic to pick
e.g. B or C, rather than A.

distlib currently contains support for the multi-valued "Provides", but I'm
not confident that will work as expected given pathological cases like the
example I suggested, without getting "complicated" in the Zen of Python sense.
I'm not convinced that the maintenance burden of a complicated solution is
worth the heretofore unnecessary ability to bundle stuff in arbitrary ways.

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

[Christian Tismer]

On 20.11.12 12:39, Nick Coghlan wrote:
On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson 
mailto:krist...@ccpgames.com>> wrote:


I’m intrigued.  I thought this was merely so that one could do

python –m mypackage.mysubpackage

Can you refer me to the rationale and discussion about this feature?


It was part of a fairly long progression in changes to the command 
line execution support :)


Top level package execution with -m came first in 2.4, arbitrary 
package execution for -m arrived in 2.5 (along with the runpy module), 
directory and zipfile execution (by supplying a valid sys.path entry 
as the "script" command line argument) was added in 2.6/3.0, and 
finally officially supported package execution via -m only arrived in 
2.7 and 3.1 (a broken version of the last accidentally existed in 2.5, 
but that was just a mistake that arose when merging the import 
emulations in runpy and pkgutil, and had a side effect that violated 
at least one of the import system invariants).


In the specific case of directory and zipfile execution, discussion 
happened on the tracker: http://bugs.python.org/issue1739468


It was never brought up on python-dev because Guido was participating 
directly in the tracker discussion. Unfortunately, we then also forgot 
to mention it in the original version of the 2.6 What's New doc, so 
the vast majority of people missed the addition :(


Hi Nick,

thank you very much for this story and the link to the issue tracker!
A very good move for Python which is really not mentioned enough
to make people more aware of a great feature.

I think part of this discussion should get a more prominent place
for gems that can be found without knowing what to search. ;-)

Is the issue tracker permanent enough to reference it?
Maybe there could be some auxiliary info page with proper keywords
that collects links to relevant discussions like this.
Do we have such a thing already?

ciao - chris

--
Christian Tismer :^)   
Software Consulting  : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

[Nick Coghlan]

On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson <
krist...@ccpgames.com> wrote:

>  I’m intrigued.  I thought this was merely so that one could do
>
> python –m mypackage.mysubpackage
>
> Can you refer me to the rationale and discussion about this feature?
>

It was part of a fairly long progression in changes to the command line
execution support :)

Top level package execution with -m came first in 2.4, arbitrary package
execution for -m arrived in 2.5 (along with the runpy module), directory
and zipfile execution (by supplying a valid sys.path entry as the "script"
command line argument) was added in 2.6/3.0, and finally officially
supported package execution via -m only arrived in 2.7 and 3.1 (a broken
version of the last accidentally existed in 2.5, but that was just a
mistake that arose when merging the import emulations in runpy and pkgutil,
and had a side effect that violated at least one of the import system
invariants).

In the specific case of directory and zipfile execution, discussion
happened on the tracker: http://bugs.python.org/issue1739468

It was never brought up on python-dev because Guido was participating
directly in the tracker discussion. Unfortunately, we then also forgot to
mention it in the original version of the 2.6 What's New doc, so the vast
majority of people missed the addition :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

[Kristján Valur Jónsson]

I'm intrigued.  I thought this was merely so that one could do
python -m mypackage.mysubpackage
Can you refer me to the rationale and discussion about this feature?

K

From: Nick Coghlan [mailto:ncogh...@gmail.com]
Sent: 18. nóvember 2012 11:25
To: Kristján Valur Jónsson
Cc: Christian Tismer; python-dev@python.org
Subject: Re: [Python-Dev] Generally boared by installation (Re: Setting project 
home path the best way)

Easily bundling dependencies is a key principle behind the ability to execute 
directories and zipfiles that contain a top level __main__.py file that was 
added back in 2.6 (although the zipfile version doesn't play nicely with 
extension modules).

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   
Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com