date:20130602

Great to hear!

Cheers,
Nick.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] A process for removal of PyPi entries

2013-06-02 Thread Richard Jones

I also support human involvement. I don't deal with so many cases that it's
such a burden - though I do have a bit of a backlog at the moment due to
lack of personal time.

Having said that, I can see value in automatically clearing out empty (no
meta-data, no files) registrations 6+ months after they're created.


On 2 June 2013 20:51,  wrote:

>
> Quoting Paul Moore :
>
>  I'm -1 on anything that doesn't involve at least a minimal level of human
>> involvement (possibly excepting an initial clean up exercise for projects
>> with no author email)
>>
>
> I support this position. This is actually how PyPI has operated over the
> last
> decade. People have always taken over projects, either the project
> entirely,
> or just the name. It always involved contacting the original owner of the
> name.
>
> In this thread, Lukas wrote
>
>
>  Fortunately we were able to work it out with Richard
>> but we had to contact him directly and waste his cycles on this.
>>
>
> I don't consider his cycles wasted at all. It's an important interaction.
>
> I'm fine with formalizing the process, and I'm also fine with adding tool
> support. However, I agree that a PEP should be written and agreed about
> this.
>
> Personally, I'd favor this procedure:
> - nothing happens unless some user explicitly requests it
> - on request, the owner is contacted, and given some time to respond
> - if they do respond, and are unwilling to yield the name, nothing
>   happens
> - if they have confirmed that they want to keep the name, they won't
>   be asked again for at least one year.
>
> Regards,
> Martin
>
>
>
>
> __**_
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> http://mail.python.org/**mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

 

 

From: Paul Moore [mailto:p.f.mo...@gmail.com] 
Sent: Sunday, 02 June, 2013 14:03



 

One other question that I can't see covered by the CHANGES and README files.
Does setuptools 0.7 support dist-info directories the same way that recent
versions of distribute did? The dist-info support is critical to wheel
support in pip.

 

 

Yes, unless otherwise specified in the merge docs, Setuptools 0.7 includes
all of Distribute.



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

Good suggestions. Historically, Setuptools has not followed semver. I intend 
for Setuptools to start to follow semver (at least in spirit) starting with 
0.7.

From: anatoly techtonik [mailto:techto...@gmail.com]
Sent: Sunday, 02 June, 2013 14:50
To: Jason R. Coombs
Cc: distutils-sig@python.org
Subject: Re: [Distutils] Setuptools 0.7 final available for download

On Sun, Jun 2, 2013 at 8:19 PM, Jason R. Coombs mailto:jar...@jaraco.com> > wrote:

See https://bitbucket.org/pypa/setuptools/src/0.7/CHANGES.txt and 
https://bitbucket.org/pypa/setuptools/src/0.7/MERGE.txt for details on the 
changes.

Thanks for the links. I expected something more massive, but as people said 
there appeared to be a plenty of releases over this period, so MERGE.txt 
covers my needs,

Major number increments mean that some features are added according to 
semver.org   and even without it it is convenient to read 
about features separately from fixes.

http://sourceforge.net/p/roundup/code/ci/default/tree/CHANGES.txt

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

2013-06-02 Thread anatoly techtonik

On Sun, Jun 2, 2013 at 8:19 PM, Jason R. Coombs  wrote:

> See https://bitbucket.org/pypa/setuptools/src/0.7/CHANGES.txt and
> https://bitbucket.org/pypa/setuptools/src/0.7/MERGE.txt for details on
> the changes.
>
Thanks for the links. I expected something more massive, but as people said
there appeared to be a plenty of releases over this period, so MERGE.txt
covers my needs,

Major number increments mean that some features are added according to
semver.org and even without it it is convenient to read about features
separately from fixes.
http://sourceforge.net/p/roundup/code/ci/default/tree/CHANGES.txt
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

2013-06-02 Thread Lennart Regebro

On Sat, Jun 1, 2013 at 5:57 PM, Jim Fulton  wrote:
> In the Python community, we've been pretty laid back
> about how we name packages.  When we were small, this made
> sense.  It doesn't make sense any more.

I don't think this is a problem, and I don't think domains or
usernames in the package names is a solution even if it is a problem.

> We should not have to come up with a process for recognizing squatters
> on simple package names.  We should have something more systematic,
> IMO.

I also don't think squatting in itself is that much of a problem. Only
once has someone been faster than me in stealing a package name and
that was "skynet". :-)
Pretty much all other package names I've ever come up with has been
free. And when somebody is squatting, I think it can be dealt with
manually, for the most time.

In fact, I'm trying to contact the skynet author now, to see if I can
get my "skynet" in there instead. ;-) [1]

Something I don't like though is the plethora of non-packages, most of
which are test packages of some sort. Just search for "foo". :-) I'd
like that to be cleaned up.

//Lennart

[1] https://github.com/regebro/skynet
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

2013-06-02 Thread Tres Seaver

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/02/2013 03:10 AM, holger krekel wrote:
> Somewhat proper import namespacing is only available with very recent
> python versions which still have a long way to become mainstream.

I don't understand this claim at all.  W'eve had packages in python for
fifteen years, and extensible namespace-package support in one form or
another for eight.  The fact that the Python 3.3 adds support for a new
spelling doesn't mean they are a new feature.


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlGrhjEACgkQ+gerLs4ltQ7jnQCfYokgj1vHL/pcun2PYmsP6EYD
ei4AoJ0AhzbIckRY+mGtIk89qXqaFjY9
=Mqzk
-END PGP SIGNATURE-

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

On Sun, Jun 2, 2013 at 1:15 PM, anatoly techtonik  wrote:
...
> Nice. This is a significant event. Is there any Changes page to see what is
> so awesome in the release that took about more than 3 years to appear
> according to PyPI page (https://pypi.python.org/pypi/setuptools)?

FTR, there have been frequent releases to setuptools over that period.
They weren't published to PyPI, but were still seen by setuptools.
(I assume through page scraping.) Not ideal, by far, I know.

There's a common misconception that setuptools has been dormant,
but that's just not accurate.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

See https://bitbucket.org/pypa/setuptools/src/0.7/CHANGES.txt and 
https://bitbucket.org/pypa/setuptools/src/0.7/MERGE.txt for details on the 
changes.

From: anatoly techtonik [mailto:techto...@gmail.com] 
Sent: Sunday, 02 June, 2013 13:15
To: Jason R. Coombs
Cc: distutils-sig@python.org
Subject: Re: [Distutils] Setuptools 0.7 final available for download

On Sun, Jun 2, 2013 at 6:42 PM, Jason R. Coombs mailto:jar...@jaraco.com> > wrote:

I’m pleased to announce the official release of Setuptools 0.7, now available 
for download from the project page  .

Nothing has changed from the 0.7b4 pre-release. This is the version that will 
be uploaded to PyPI after we work out the technique to deploy to PyPI without 
interfering with the setuptools 0.6 releases.

Nice. This is a significant event. Is there any Changes page to see what is so 
awesome in the release that took about more than 3 years to appear according to 
PyPI page (https://pypi.python.org/pypi/setuptools)?

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] A process for removal of PyPi entries

2013-06-02 Thread Paul Moore

On 2 June 2013 12:54, Donald Stufft  wrote:

> The missing case here is what happens if they don't respond?


That (and when there is no author contact details) is when a unilateral
process *is* needed. In those cases, I'd argue that we should have some
overall guidelines to work from, but ultimately the PyPI admin(s) should
decide on a case by case basis.

If and when the volume of such requests gets so great that the admins are
overwhelmed, *then* we should look at the feasibility of an automatic
process.
Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

Hi Paul.

 

From: Paul Moore [mailto:p.f.mo...@gmail.com] 
Sent: Sunday, 02 June, 2013 13:13

 

On 2 June 2013 16:42, Jason R. Coombs mailto:jar...@jaraco.com> > wrote:

For convenience, I've also added experimental .msi installers for Windows
for Python 3.3 and Python 2.7. Work may continue on these in the future, but
as the documentation states, the recommended installation procedure is to
use ez_setup.py.


Please consider using wheel instead of msi. Or at least bdist_wininst. The
msi format is opaque and cannot be converted to other formats. OTOH,
setuptools is pure Python, so having binaries is a relative non-issue.

 

Thanks. I need to learn wheel. I haven't yet done that. The reason I chose
msi is because it is a binary distribution format that works on 64-bit
Python (there still exists a bug in distutils where installers don't detect
64-bit python installations because the installer executable is 32-bit).

 

The recommended installation technique on all platforms, much like
Distribute, is to do a source install.

 

One point - can I assume that the new version is written to run unchanged on
all supported Python versions (2 and 3) so that it is possible to build a
wheel using *any* version of Python and use it unchanged on any other? (I
ask because I'd like to look at integrating setuptools 0.7 into virtualenv).

 

Currently, Python 3 support is still using the same technique employed by
distribute - namely, running 2to3 during install. So I don't believe it can
be used unchanged on any Python version. Having a single code base for
Python 2.4-3.3 is the top feature I want to develop as soon as the merge is
complete and successfully adopted.

 



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

2013-06-02 Thread anatoly techtonik

On Sun, Jun 2, 2013 at 6:42 PM, Jason R. Coombs  wrote:

> I’m pleased to announce the official release of Setuptools 0.7, now
> available for download from the project 
> page
> .
>
> ** **
>
> Nothing has changed from the 0.7b4 pre-release. This is the version that
> will be uploaded to PyPI after we work out the technique to deploy to PyPI
> without interfering with the setuptools 0.6 releases.
>

Nice. This is a significant event. Is there any Changes page to see what is
so awesome in the release that took about more than 3 years to appear
according to PyPI page (https://pypi.python.org/pypi/setuptools)?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download


On Jun 2, 2013, at 11:42 AM, "Jason R. Coombs"  wrote:

> I’m pleased to announce the official release of Setuptools 0.7, now available 
> for download from the project page.
>  
> Nothing has changed from the 0.7b4 pre-release. This is the version that will 
> be uploaded to PyPI after we work out the technique to deploy to PyPI without 
> interfering with the setuptools 0.6 releases.
>  
> For convenience, I’ve also added experimental .msi installers for Windows for 
> Python 3.3 and Python 2.7. Work may continue on these in the future, but as 
> the documentation states, the recommended installation procedure is to use 
> ez_setup.py.
>  
> To install this latest release, follow the canonical install instructions 
> (using ez_setup.py), but direct the script to use bitbucket instead of PyPI:
>  
> python ./ez_setup.py 
> --download-base=https://bitbucket.org/pypa/setuptools/downloads/
>  
> I’ll send another announcement when the official release has been uploaded to 
> PyPI and the ez_setup.py script can be used directly.
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

Great work!

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 0.7 final available for download

I'm pleased to announce the official release of Setuptools 0.7, now
available for download from the project page
 .

 

Nothing has changed from the 0.7b4 pre-release. This is the version that
will be uploaded to PyPI after we work out the technique to deploy to PyPI
without interfering with the setuptools 0.6 releases.

 

For convenience, I've also added experimental .msi installers for Windows
for Python 3.3 and Python 2.7. Work may continue on these in the future, but
as the documentation states, the recommended installation procedure is to
use ez_setup.py.

 

To install this latest release, follow the canonical install instructions
(using ez_setup.py), but direct the script to use bitbucket instead of PyPI:

 

python ./ez_setup.py
--download-base=https://bitbucket.org/pypa/setuptools/downloads/

 

I'll send another announcement when the official release has been uploaded
to PyPI and the ez_setup.py script can be used directly.



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.


On Jun 2, 2013, at 11:25 AM, Jim Fulton  wrote:

> On Sat, Jun 1, 2013 at 3:21 PM, holger krekel  wrote:
>> On Sat, Jun 01, 2013 at 11:57 -0400, Jim Fulton wrote:
>>> In the Python community, we've been pretty laid back
>>> about how we name packages.  When we were small, this made
>>> sense.  It doesn't make sense any more.
>> 
>> I've heart this sentiment before, but would like to read more
>> clearly stated problems.
> 
> I thought the problem was pretty clear: name collisions.
> 
> There's a parallel thread on how to detect and reclaim
> names that are taken and unused.  I think if we had a more
> systematic way of naming packages, this wouldn't be an issue.

I think the parallel thread mostly involves people wanting specific names
that are no longer available. I know I've claimed a few good names at the
start of an idea then abandoned the idea and forgotten to deregister the name.

There are still lots of good names but people get attached to ones they like
and it feels kinda shitty when you have to rename your project because of
a placeholder that's been there for 3 years that you didn't notice.

> 
>>> We should not have to come up with a process for recognizing squatters
>>> on simple package names.  We should have something more systematic,
>>> IMO.
>>> 
>>> Unfortunately, I think the sanest way of avoiding most package
>>> name issues is to base them on domains, as is done in the Java
>>> world.  This goes against the Python philosophy of preferring
>>> flat to nested, but I still think it's better than trying to police 
>>> squatters,
>>> or to encouraging races to claim top-level names.
>> 
>> I am not sure that tying to DNS namespacing is the only solution here
>> (whatever the problem is exactly :).
> 
> It's not the only solution, but it's a pretty easy one.  It avoids (more
> more precisely reuses) a central naming authority. It's the technique
> used by the java ecosystem and by XML namespaces, for example.

We already have a central naming authority it's called PyPI ;)

> 
> 
>> 
>>> For a while, many of us have been pretty careful to use namespaces
>>> for new packages to mitigate this issue.  For example, the zc namespace
>>> is a shorter version of com.zope, but at some point, it won't be fair
>>> for us to claim zc for ourselves.
>> 
>> I wonder if we could allow people/groups to apply (to humans) for a
>> namespace which they can subsequently control, like the "zc.*" one.
> 
> We could.  Maybe that's the most palatable alternative.  It has the
> huge benefit of promoting relatively flat, Pythonic, namespaces.
> 
> It has disadvantages:
> 
> - We have to set up some sort of naming authority.
> 
> - It's probably not scalable.

As mentioned above we already have an authority, and it's more or less scaled
quite fine so far. I still continue to be able to get very good names, but 
sometimes
it takes a few tries to find an unused one.

> 
>> Everyone could continue to push non-namespaced (flat) packages to pypi
>> like now but the names couldn't take the form of namespaced ones.
> 
> I'm not sure what you're suggesting here.
> 
> Are you saying someone could publish a package named: "zc",
> bit not "zc.foo"?
> 
> Or are you saying that publication of a package named "bar" would
> prevent someone from creating a "bar" namespace, and the
> other way around?

This is likely implementation details, but if I were to do it then folks could
apply for a particular namespace, (For example zc.*) and it would give
them ownership over all levels of that namespace, (in our example, zc,
zc.foo, zc.bar, zc.foo.bar, etc but not zc4 or zc4.cool).

However they would only own that namespace if they've applied and been
granted it. Otherwise anyone is free to register packages in that namespace.

And for what it's worth Organization accounts is something on my roadmap for
PyPI.

> 
> Jim
> 
> -- 
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

On Sat, Jun 1, 2013 at 6:00 PM, Paul Moore  wrote:
>
> On 1 June 2013 16:57, Jim Fulton  wrote:
>> In the Python community, we've been pretty laid back
>> about how we name packages.  When we were small, this made
>> sense.  It doesn't make sense any more.
>
> I'd like to see some evidence that this is the case.

How about the "A process for removal of PyPi entries" thread?

> It doesn't seem so to
> me - most package names are relatively discoverable and/or intuitive,

Um, boto?  fabric? paramiko? kazoo? The way I find packages is by searching.
Names are irrelevant.

> and we
> currently have basically no namespacing.

We have a ton of namespacing.  It's just informal.  IMO, it's
a result of good hygiene and citizenship (I don't mean to
dis anyone not using namespaces).

> There's a long way to go before
> something like your suggestion is needed, in my view.

  If the projects now using namespaces weren't, I
predict the problem would be a lot more apparent.

>
>> Unfortunately, I think the sanest way of avoiding most package
>> name issues is to base them on domains, as is done in the Java
>> world.  This goes against the Python philosophy of preferring
>> flat to nested, but I still think it's better than trying to police
>> squatters,
>> or to encouraging races to claim top-level names.
>
>
> No, no, no...
>
> There's the point Donald made that you require people to own a domain (or
> you create some sort of hack like
> org.bitbucket.username/com.github.username/...) but it also makes package
> names unreasonably deep, and requires an explosion of namespace packages at
> the top level. And it's ugly :-)
>
> Perl manages with a relatively flat namespace and relatively informal rules
> for managing package names (AIUI). I'm sure Python can, too.

Perl's a dead language. :)

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

On Sat, Jun 1, 2013 at 4:13 PM, Jason R. Coombs  wrote:
>
>
>
>
> From: Distutils-SIG
> [mailto:distutils-sig-bounces+jaraco=jaraco@python.org] On Behalf Of
> Donald Stufft
> Sent: Saturday, 01 June, 2013 15:30
> To: holger krekel
> Cc: distutils sig
> Subject: Re: [Distutils] Sooner or later, we're going to have to be more
> formal about how we name packages.
>
>
>
>
>
> On Jun 1, 2013, at 3:21 PM, holger krekel  wrote:
>
>
>
> On Sat, Jun 01, 2013 at 11:57 -0400, Jim Fulton wrote:
>
>
> For a while, many of us have been pretty careful to use namespaces
> for new packages to mitigate this issue.  For example, the zc namespace
> is a shorter version of com.zope, but at some point, it won't be fair
> for us to claim zc for ourselves.
>
>
> I wonder if we could allow people/groups to apply (to humans) for a
> namespace which they can subsequently control, like the "zc.*" one.
>
>
>
> So for example if the django community wants to introduce the concept
>
>
> of "vetted" plugins/addons, they could move to manage "dj.*" or so.
>
>
>
> I think this example highlights some of the challenges with
> registering/controlling namespaces – who owns what and what is the meaning
> of a (distribution) package name? For example, what is the namespace used
> for an endorsed django plugin written by zope corporation?

IMO the purpose of the namespace is to organize names.  Whoever owns the
namespace decides what names can go into it.  It's purely a name management
issue, not, for example, a intellectual property issue.

If Zope Corporation independently creates a Django plugin, I'd expect it to
go into the "zc" namespace.  OTOH, Zope Corporation was an active
member of the Django community, it might publish to the "dj" (or whatever)
namespace, or request permission to do so.

I started using the zc namespace a few years ago because I didn't want
to impose our names on the Zope community.

> I think more people would claim namespaces when namespaces are better
> supported in Python. My expectation is Python 3.3 namespace package support
> will ease that challenge (when it becomes a dominant version).

This is somewhat baffling to me.  We've used namespaces for over a decade
virtually without issue.

(We;ve used namespaces far longer than that, going all the way back to "ni"
 with relatively minor issues.)

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

On Sat, Jun 1, 2013 at 3:21 PM, holger krekel  wrote:
> On Sat, Jun 01, 2013 at 11:57 -0400, Jim Fulton wrote:
>> In the Python community, we've been pretty laid back
>> about how we name packages.  When we were small, this made
>> sense.  It doesn't make sense any more.
>
> I've heart this sentiment before, but would like to read more
> clearly stated problems.

I thought the problem was pretty clear: name collisions.

There's a parallel thread on how to detect and reclaim
names that are taken and unused.  I think if we had a more
systematic way of naming packages, this wouldn't be an issue.

>> We should not have to come up with a process for recognizing squatters
>> on simple package names.  We should have something more systematic,
>> IMO.
>>
>> Unfortunately, I think the sanest way of avoiding most package
>> name issues is to base them on domains, as is done in the Java
>> world.  This goes against the Python philosophy of preferring
>> flat to nested, but I still think it's better than trying to police 
>> squatters,
>> or to encouraging races to claim top-level names.
>
> I am not sure that tying to DNS namespacing is the only solution here
> (whatever the problem is exactly :).

It's not the only solution, but it's a pretty easy one.  It avoids (more
more precisely reuses) a central naming authority. It's the technique
used by the java ecosystem and by XML namespaces, for example.

>
>> For a while, many of us have been pretty careful to use namespaces
>> for new packages to mitigate this issue.  For example, the zc namespace
>> is a shorter version of com.zope, but at some point, it won't be fair
>> for us to claim zc for ourselves.
>
> I wonder if we could allow people/groups to apply (to humans) for a
> namespace which they can subsequently control, like the "zc.*" one.

We could.  Maybe that's the most palatable alternative.  It has the
huge benefit of promoting relatively flat, Pythonic, namespaces.

It has disadvantages:

- We have to set up some sort of naming authority.

- It's probably not scalable.

> Everyone could continue to push non-namespaced (flat) packages to pypi
> like now but the names couldn't take the form of namespaced ones.

I'm not sure what you're suggesting here.

Are you saying someone could publish a package named: "zc",
bit not "zc.foo"?

Or are you saying that publication of a package named "bar" would
prevent someone from creating a "bar" namespace, and the
other way around?

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

On Sat, Jun 1, 2013 at 2:54 PM, Noah Kantrowitz  wrote:
>
> On Jun 1, 2013, at 11:09 AM, Jim Fulton wrote:
>
>> On Sat, Jun 1, 2013 at 2:02 PM, Donald Stufft  wrote:
>>>
>>> On Jun 1, 2013, at 2:01 PM, Donald Stufft  wrote:
>>>
>>>
>>> I am opposed to this. Requiring someone to have purchased a domain adds a
>>> significant
>>> to publishing a project. If there are no requirements that they have
>>> purchased the domain
>>> then it's nothing more than a convention and something that anyone who wants
>>> to do
>>> this can do.
>>
>> Fair enough. A common variation on this scenario, which avoids
>> purchasing a domain,
>> is to use a code hosting domain and project name, so, for example:
>> org.bitbucket.j1m.foo.
>>
>> Of course, using a domain name without owning it is a form of squatting.
>
> All that means is either we move the problem (instead of one shared namespace 
> we two or three common ones)

I don't understand why you say two or three. There would be as many
namespaces as there are
domains or VCS accounts.  There would be many distinct namespaces,
each controlled controlled
by a single user or organization.

> or we do it github-style and just prepend usernames at which point you can 
> skip the whole URI thing because usernames must be unique for reasons of 
> general sanity and I don't think it is a huge deal that a single person can't 
> have two packages of the same name.

That's an option. (I assume you mean PyPI user names.) It would be
more attractive if PyPI supported
organizational accounts. (I sure wish it did.)  I can't say I find the
idea of tying a package name to an
account name attractive, but it's a good alternative for projects
without a domain.

> Github-style namespacing just means that either names all suck 
> (django/django, kennethreitz/requests) or you need to come up with some way 
> to map un-namespaced names to their canonical form and we are more or less 
> back at square one. If people don't mind the sucky names, they can already 
> put that in their package name if the bare version is taken, so QED this is 
> already doable in the current system, it just looks so ugly that no one wants 
> to do it and enforcing the ugly seems like a poor option.

My observation of the java world is that most packages that get
published to central repositories end
up having domain based names.  Even that sucks to some degree because
flat is better than nested.
I just don't think the current ad-hoc mechanism we're using now is scalable.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

2013-06-02 Thread Trishank Karthik Kuppusamy


On 6/2/13 9:01 AM, Nick Coghlan wrote:

On Sun, Jun 2, 2013 at 10:09 PM, Donald Stufft  wrote:

If we deploy some sort of end to end signing I think TUF is a good
implementation of it.

I'm not sold on the possibility of reasonably doing end to end signing here
though.


I think in the long run it's a technology we want to offer, but even
with it deployed PyPI would continue to act as a trusted intermediary
in most cases. Effective key management is such a PITA that only a few
larger projects would be in a real position to take direct advantage
of end-to-end signing - for the remaining projects, trusting PyPI not
to get compromised is already the status quo.



Yes, key management could be a real PITA if we do not consider 
usability. In our design proposal, we talked about how to try to 
maximize usability and security, by keeping the truly critical keys 
offline (which would be used rarely), and the not-so-critical keys 
online (which means that automation can easily use them).


We will be working on TUF and PyPI full-time this summer. As I write 
this, we are introducing additional security mechanisms for some cases 
which arise frequently; e.g. how do we tell TUF to put more trust in 
packages from a stable-release role versus a bleeding-edge role?


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

2013-06-02 Thread Trishank Karthik Kuppusamy

On 6/2/13 4:21 AM, Nick Coghlan wrote:

On Sun, Jun 2, 2013 at 5:37 PM, holger krekel wrote:

Speaking of TUF: is there some kind of PEP like doc floating already?

Just the proof-of-concept the TUF folks created about using it to
secure /simple. I'm personally sold on the technology itself as
something we should deploy in the long run, but I think it makes sense
to wait until we have the static dependency metadata publication and
various other PyPI related infrastructure issues sorted out before we
try to offer additional protection above and beyond trusting the SSL
CA system and PyPI itself.

That said, one of the reasons PEP 426 calls out the "essential
dependency resolution" fields is that those are the ones I think it
may make sense to embed in the TUF custom metadata fields.

Nick got our proof-of-concept pretty much right, and I just want to make
this correction: we offered security for both /simple and /packages, but
only for a subset of packages. We were working on securing all the
packages under PyPI, but were derailed by some projects with immediate
deadlines.

The good news is that we will be continuing our work full-time this
summer, and expect to make much progress.

We don't have a PEP for it, besides our design proposal[1]. I think a
PEP is a good idea, and we should draft one along the way.

[1]
https://docs.google.com/document/d/1sHMhgrGXNCvBZdmjVJzuoN5uMaUAUDWBmn3jo7vxjjw/edit?usp=sharing

___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

On Sun, Jun 2, 2013 at 10:09 PM, Donald Stufft  wrote:
> If we deploy some sort of end to end signing I think TUF is a good
> implementation of it.
>
> I'm not sold on the possibility of reasonably doing end to end signing here
> though.

I think in the long run it's a technology we want to offer, but even
with it deployed PyPI would continue to act as a trusted intermediary
in most cases. Effective key management is such a PITA that only a few
larger projects would be in a real position to take direct advantage
of end-to-end signing - for the remaining projects, trusting PyPI not
to get compromised is already the status quo.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.


On Jun 2, 2013, at 4:21 AM, Nick Coghlan  wrote:

> On Sun, Jun 2, 2013 at 5:37 PM, holger krekel  wrote:
>> Speaking of TUF: is there some kind of PEP like doc floating already?
> 
> Just the proof-of-concept the TUF folks created about using it to
> secure /simple. I'm personally sold on the technology itself as
> something we should deploy in the long run, but I think it makes sense
> to wait until we have the static dependency metadata publication and
> various other PyPI related infrastructure issues sorted out before we
> try to offer additional protection above and beyond trusting the SSL
> CA system and PyPI itself.
> 
> That said, one of the reasons PEP 426 calls out the "essential
> dependency resolution" fields is that those are the ones I think it
> may make sense to embed in the TUF custom metadata fields.
> 
> Cheers,
> Nick.
> 
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


If we deploy some sort of end to end signing I think TUF is a good 
implementation of it.

I'm not sold on the possibility of reasonably doing end to end signing here 
though.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] A process for removal of PyPi entries


On Jun 2, 2013, at 6:51 AM, mar...@v.loewis.de wrote:

> 
> Quoting Paul Moore :
> 
>> I'm -1 on anything that doesn't involve at least a minimal level of human
>> involvement (possibly excepting an initial clean up exercise for projects
>> with no author email)
> 
> I support this position. This is actually how PyPI has operated over the last
> decade. People have always taken over projects, either the project entirely,
> or just the name. It always involved contacting the original owner of the 
> name.
> 
> In this thread, Lukas wrote
> 
>> Fortunately we were able to work it out with Richard
>> but we had to contact him directly and waste his cycles on this.
> 
> I don't consider his cycles wasted at all. It's an important interaction.
> 
> I'm fine with formalizing the process, and I'm also fine with adding tool
> support. However, I agree that a PEP should be written and agreed about this.
> 
> Personally, I'd favor this procedure:
> - nothing happens unless some user explicitly requests it
> - on request, the owner is contacted, and given some time to respond
> - if they do respond, and are unwilling to yield the name, nothing
>  happens
> - if they have confirmed that they want to keep the name, they won't
>  be asked again for at least one year.

The missing case here is what happens if they don't respond?

> 
> Regards,
> Martin
> 
> 
> 
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] A process for removal of PyPi entries

2013-06-02 Thread martin



Quoting Paul Moore :


I'm -1 on anything that doesn't involve at least a minimal level of human
involvement (possibly excepting an initial clean up exercise for projects
with no author email)


I support this position. This is actually how PyPI has operated over the last
decade. People have always taken over projects, either the project entirely,
or just the name. It always involved contacting the original owner of  
the name.


In this thread, Lukas wrote


Fortunately we were able to work it out with Richard
but we had to contact him directly and waste his cycles on this.


I don't consider his cycles wasted at all. It's an important interaction.

I'm fine with formalizing the process, and I'm also fine with adding tool
support. However, I agree that a PEP should be written and agreed about this.

Personally, I'd favor this procedure:
- nothing happens unless some user explicitly requests it
- on request, the owner is contacted, and given some time to respond
- if they do respond, and are unwilling to yield the name, nothing
  happens
- if they have confirmed that they want to keep the name, they won't
  be asked again for at least one year.

Regards,
Martin



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

On Sun, Jun 2, 2013 at 5:37 PM, holger krekel  wrote:
> Speaking of TUF: is there some kind of PEP like doc floating already?

Just the proof-of-concept the TUF folks created about using it to
secure /simple. I'm personally sold on the technology itself as
something we should deploy in the long run, but I think it makes sense
to wait until we have the static dependency metadata publication and
various other PyPI related infrastructure issues sorted out before we
try to offer additional protection above and beyond trusting the SSL
CA system and PyPI itself.

That said, one of the reasons PEP 426 calls out the "essential
dependency resolution" fields is that those are the ones I think it
may make sense to embed in the TUF custom metadata fields.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

2013-06-02 Thread holger krekel

On Sun, Jun 02, 2013 at 17:26 +1000, Nick Coghlan wrote:
> On Sun, Jun 2, 2013 at 5:10 PM, holger krekel  wrote:
> > If pypi has no idea about namespaces (like i considered them in my other 
> > post)
> > then using namespaces do not really provide much.  Someone can still come 
> > along
> > and publish within that pseudo-namespace.  I would think the goal of
> > pypi-namespaces would be to give a group control over anything that's
> > released using it, allowing to communicate install-users certain guarantees.
> >
> > However, before further discussion i think there first needs to be more
> > reasoning and stating of practical problems with the current
> > anyone-can-register-anything-that's-not-taken model.
> 
> TUF actually has native support for prefix delegation, but actually
> *using* that is a long way down the todo list at the moment. Static
> dependency metadata publication and end-to-end signature support are
> well ahead of it and will likely keep us collectively busy for a while
> yet.

No worries, I understood already that it's not high on your list.  

I'd appreciate, however, if Jim or someone else could state the problems
with missing namespacing so we can start a discussion from there later.

Speaking of TUF: is there some kind of PEP like doc floating already?

cheers,
holger

> Cheers,
> Nick.
> 
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> 
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [ANN] pypiserver 1.1.1 - minimal private pypi server

2013-06-02 Thread holger krekel

On Sat, Jun 01, 2013 at 23:26 +0200, Ralf Schmitt wrote:
> holger krekel  writes:
> 
> >> How does it compare to http://pypi.python.org/pypi/devpi-server?
> >
> > the main differences as i see them (note i am the devpi-server author, 
> > though):
> >
> > - pypiserver supports uploading to private indexes, 
> >   devpi-server not yet (but next week / on trunk already :)
> 
> I still consider that a bug. scp works much better. I'm not sure if
> devpi-server maintains some kind of database, but I consider it a big
> plus to be able to just copy files and directories to the package
> directory and serve them instantly.

(devpi-server maintains a database, indeed).

If you scp a large file and someone else is about the pip-install that
package, could you get partial downloads which fail on the user side?

I like the ability of transfering things via scp but i guess in devpi-server
i'd make them go to a "clearing area" where they are screened before put
up for serving.

> ...
> > - pypiserver redirects the lookup of pypi packages to pypi.python.org,
> 
> that's related to a weak point of pypiserver: you currently have to
> parse the log files and look for HTTP redirects if you like to know what
> packages are missing in the local package directory.

It's a recursive problem because those packages might have pulled in
yet more dependencies.

> you also can't install a specific version of a package if the package
> directory doesn't have that specific version but a different version.
> this may or may not be good depending on your use case.
> 
> redirects can also be completely disabled, so pip/easy_install only see
> the packages in the package directory.

FYI devpi-server supports multiple indexes and you can define inheritance 
semantics between indexes.  If an index A inherits from another index B
(which can be the full pypi-mirroring index) then queries on A will
see all release files from index B merged in.  By default, there also
is a "root/prod" index which does not inherit anything and contains
only the packages explicitely pushed to it.  If you install from such
a no-bases index, only its own list of release files are considered.

This is designed such that users can start off with "play areas" to prepare
and test their packages including third party ones before "staging" them 
to a more constrained no-bases index.

> >   devpi-server caches them and serves everything (including #egg-links)
> >   through itself, allowing complete offline operations (including
> >   caching/serving of 3rd party site's packages) using the 
> >   extended PEP381 mirror protocol
> >
> > - pypiserver has a good and simple implementation, devpi-server is
> >   little more complex mostly due to its caching/crawling logic.
> >
> > - both are very well tested and maintained but pypiserver is out there
> >   for a longer time already, so has seen more RL-testing.
> >
> > Ralph, please add/comment as you see fit.
> 
> pypiserver exhibits it's WSGI application and might be easier to deploy
> using different WSGI servers.

In principle, A wsgi-app is also available with devpi-server but there is
one issue i am unsure about: the wsgi-app needs to have async threads/greenlets
running to keep the pypi-mirror cache up-to-date.  Starting those threads
at wsgi-app creation seems too early so it currently starts them before
bottle.run() is invoked.  I guess it could also happen at "first-request"
time or so which should make the creation of the WSGI-app become a
no-sideeffect operation.

cheers,
holger

> -- 
> Cheers
> Ralf
> 
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.