Re: Thoughts on Kik and NPM and implications for CPAN

[Philippe Bruhat (BooK)]

On Thu, Mar 24, 2016 at 05:38:13PM -0400, Olaf Alders wrote:
> 
> > On Mar 24, 2016, at 5:02 PM, Neil Bowers  wrote:
> > 
> >>> PAUSE doesn’t (currently) know the river position, but if it published
> >>> a feed of deletion-schedulings, then some third-party agent could
> >>> monitor the feed and check for dists that are on river. I think those
> >>> are the dists that should be alerted to modules@ […] Obviously the
> >>> issue here is DarkPAN: a dist might not have any CPAN dependents, but
> >>> may be used plenty out in the big bad world. That’s a separate problem
> >>> :-)
> >> 
> >> I don’t think so. Plack::Middleware::Rewrite is used by a ton of people
> >> and klaxons certainly ought to ring if I ever opened up that namespace.
> >> The number of on-CPAN dependents is just 3 though.
> > 
> > The key word in what I said was *any*. I think even 3 dependents should 
> > klaxon.
> > Plus having any favourites on CPAN should also prompt the klaxon as well: I 
> > use favourites as a proxy for “has dependents” in both the adoption list 
> > and weighting dists for the PRC, and it seems to work.
> > 
> > I still think we need a service where you can say “I’m using this dist”. I 
> > think I’ll add that feature to the dashboard, which I’ll be working on at 
> > the QAH.
> 
> This would be pretty easy to bolt onto MetaCPAN.  I was already considering 
> something like this to run parallel to ++ where ++ means "I recommend this" 
> and there's some alternate symbol for saying "I use this".  This would make 
> it easy to have a script that would scan deps in apps and add them to your "I 
> use this" list in MetaCPAN.

Years ago, Léon Brocard (I think) published a script that did basically
explore your disk looking for use lines, and reported them for publication
on some dash/leaderboard.

A button for manual addition is a good first step, but something automated
might give more thorough results.

OK, I did a bit of search on use.perl.org, and I found these:
- http://use.perl.org/use.perl.org/_acme/journal/10432.html
- http://use.perl.org/use.perl.org/_acme/journal/10623.html

The service is down, but the Internet Archive has some old copies:
https://web.archive.org/web/20041010044220/http://www.astray.com/cpanstats/

> Then, if you're planning on making a controversial change to module Y,
> you have a list of users whom you can warn or poll for advice.

Preventing anonymous posts would also prevent some popularity contest
and ballot stuffing.

-- 
 Philippe Bruhat (BooK)

 Just because you do not see it does not mean it is not there.
(Moral from Groo The Wanderer #85 (Epic))

Re: Thoughts on Kik and NPM and implications for CPAN

[Kent Fredric]

That scenario doesn't seem right. A mere deletion of a .pm file in a future
release aught to be the tripwire for such a warning. An explicit namespace
clearance is much more dire.
On 25/03/2016 11:51, "Neil Bowers"  wrote:

> I wonder what the volume in one case vs the other is. Maybe the attempt
> to distinguish the cases is premature optimisation that can be skipped?
> (Hopefully so, but I don’t know.)
>
>
> I suspect you’re right, and we should start off with everything, and only
> worry if it seems too noisy.
>
> One optimisation we might consider including from the start:
>
> Alert if an indexed release is scheduled for deletion, unless there’s a
> higher version of the same dist already indexed.
>
>
> This would prevent the klaxon going off in the case where Foo-Bar-1.01
> included Foo::Bar::Error, which was changed to Foo::Bar::Exception in 1.02.
> With both releases in your author directory, both Foo-Bar-1.01 and
> Foo-Bar-1.02 will appear in 02packages, with Foo-Bar-1.01 only appearing
> against Foo::Bar::Error.
>
> Thinking about it, it should probably still be alerted on, just on the
> off-chance that the rug *is* getting pulled from under someone else, but
> it could be flagged as this “possible module renaming” case.
>
> Neil
>
>

Re: Thoughts on Kik and NPM and implications for CPAN

[Olaf Alders]

> On Mar 24, 2016, at 5:02 PM, Neil Bowers  wrote:
> 
>>> PAUSE doesn’t (currently) know the river position, but if it published
>>> a feed of deletion-schedulings, then some third-party agent could
>>> monitor the feed and check for dists that are on river. I think those
>>> are the dists that should be alerted to modules@ […] Obviously the
>>> issue here is DarkPAN: a dist might not have any CPAN dependents, but
>>> may be used plenty out in the big bad world. That’s a separate problem
>>> :-)
>> 
>> I don’t think so. Plack::Middleware::Rewrite is used by a ton of people
>> and klaxons certainly ought to ring if I ever opened up that namespace.
>> The number of on-CPAN dependents is just 3 though.
> 
> The key word in what I said was *any*. I think even 3 dependents should 
> klaxon.
> Plus having any favourites on CPAN should also prompt the klaxon as well: I 
> use favourites as a proxy for “has dependents” in both the adoption list and 
> weighting dists for the PRC, and it seems to work.
> 
> I still think we need a service where you can say “I’m using this dist”. I 
> think I’ll add that feature to the dashboard, which I’ll be working on at the 
> QAH.

This would be pretty easy to bolt onto MetaCPAN.  I was already considering 
something like this to run parallel to ++ where ++ means "I recommend this" and 
there's some alternate symbol for saying "I use this".  This would make it easy 
to have a script that would scan deps in apps and add them to your "I use this" 
list in MetaCPAN.

Then, if you're planning on making a controversial change to module Y, you have 
a list of users whom you can warn or poll for advice.

Olaf

Re: Thoughts on Kik and NPM and implications for CPAN

[Neil Bowers]

>> PAUSE doesn’t (currently) know the river position, but if it published
>> a feed of deletion-schedulings, then some third-party agent could
>> monitor the feed and check for dists that are on river. I think those
>> are the dists that should be alerted to modules@ […] Obviously the
>> issue here is DarkPAN: a dist might not have any CPAN dependents, but
>> may be used plenty out in the big bad world. That’s a separate problem
>> :-)
> 
> I don’t think so. Plack::Middleware::Rewrite is used by a ton of people
> and klaxons certainly ought to ring if I ever opened up that namespace.
> The number of on-CPAN dependents is just 3 though.

The key word in what I said was *any*. I think even 3 dependents should klaxon.
Plus having any favourites on CPAN should also prompt the klaxon as well: I use 
favourites as a proxy for “has dependents” in both the adoption list and 
weighting dists for the PRC, and it seems to work.

I still think we need a service where you can say “I’m using this dist”. I 
think I’ll add that feature to the dashboard, which I’ll be working on at the 
QAH.

Neil

Re: Thoughts on Kik and NPM and implications for CPAN

[Leon Timmermans]

On Wed, Mar 23, 2016 at 4:07 PM, David Golden  wrote:

> If you don't know what I'm referring to, read
> http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
>
> Leaving aside the IP issue, I think it might be worth considering what
> would currently happen if someone chose a 'mass removal' and whether that's
> what we'd like to have happen.
>
> N.B. this is more extreme than
> http://www.xenoterracide.com/2015/05/abandoning-all-perl-modules.html --
> that dropped perms, but left the tarballs indexed.  What if someone goes
> beyond that...
>
> Consider a scenario for user "Pat":
> * Pat schedules all tarballs for deletion and waits 3 days
> * All tarballs are deleted by PAUSE
> * mldistwatch de-indexes any previously indexed tarballs
> * Pat removes all comaints for all modules
> * Pat drops primary permissions on all modules
> * Pat drops co-maint perms on all modules
>
>
> At that point, anything depending on Pat's tarballs is broken, as they
> aren't indexed (ignoring for the moment cpanm's use of backpan indexes).
>
> Also, I think the next tarball uploaded with a namespace previously
> controlled by Pat gets "first come" permissions and is indexed (regardless
> of version number).
>
> Have I got that scenario right?
>
> My thoughts:
>
> * I think we have to allow mass deletion, even if that de-indexes stuff.
> I think that's an author's right.
>
> * I think we should *not* free up namespaces for random takeover
>
> * I think PAUSE admins should consider a reasonable request by a
> responsible-seeming party to take over a namespace (e.g. by forking a
> tarball from BackPAN).
>
> In other words: authors own their tarballs, but PAUSE owns the namespaces
> (and periodically delegates responsibility to a maintainer).
>
> Mechanically, I think that means that when PAUSE is dropping permissions,
> it should instead transfer control to a PAUSE-controlled ID.  (Effectively,
> https://github.com/andk/pause/issues/169 )
>
> Thoughts?
>

Making "give up first-come" instead be either a "donate to a co-maint or to
ADOPTME" would make sense to me.

Increasing the deletion time for indexed dists may also make sense, but
given people can upload a bogus new version it shouldn't be done too
annoying.

Leon

Re: Thoughts on Kik and NPM and implications for CPAN

[Neil Bowers]

> However, we (the CPAN community) can do a lot of things after that to 
> mitigate any damage. I wholeheartedly agree with transferring namespace 
> permissions to something that the PAUSE admins control, so any random joe 
> cannot claim the namespace and upload whatever he likes into it (this is an 
> attack vector we must keep closed).  We also need to be able to act quickly 
> to publish something in its place so installations pulling directly from the 
> CPAN do not break.  I would suggest an email alert go out to the modules@ 
> list (or another list, should this prove too noisy) providing notification 
> that an indexed module is being deleted and de-indexed.

I’ve got no idea what the monthly volume of deletions is, but I think there are 
two main cases:

1. dists that aren’t used by anything else
2. dists that are somewhere on the river

It would be nice to have a feed of all dists scheduled for deletion, as soon as 
they’re scheduled, with additional alerting if the dist is upriver at all.
PAUSE doesn’t (currently) know the river position, but if it published a feed 
of deletion-schedulings, then some third-party agent could monitor the feed and 
check for dists that are on river. I think those are the dists that should be 
alerted to modules@

Even if all deletions go to modules@, it would still be handy if that 
notification mentioned river position. Maybe PAUSE could publish an hourly list 
of files that are currently scheduled for deletion, similar to various other 
files it generates?

Obviously the issue here is DarkPAN: a dist might not have any CPAN dependents, 
but may be used plenty out in the big bad world. That’s a separate problem :-)

Neil

Re: Thoughts on Kik and NPM and implications for CPAN

[David Golden]

On Thu, Mar 24, 2016 at 10:26 AM, Neil Bowers 
wrote:

>
> It would be nice to have a feed of all dists scheduled for deletion, as
> soon as they’re
>

I really only want email for *indexed* distributions, not for deletion of
old or trial dists.  (Could you imagine the traffic on CPAN cleanup day?)
But adding river position would be a nice touch.

If someone wants to have all deletions go into an RSS feed or whatever,
that's fine with me because it won't clog my inbox.

David


-- 
David Golden  Twitter/IRC/Github: @xdg

Re: Thoughts on Kik and NPM and implications for CPAN

[Karen Etheridge]

Done: https://github.com/andk/pause/issues/204

On Wed, Mar 23, 2016 at 10:57 AM, David Golden  wrote:

> On Wed, Mar 23, 2016 at 1:39 PM, Karen Etheridge  wrote:
>
>> I would suggest an email alert go out to the modules@ list (or another
>> list, should this prove too noisy) providing notification that an indexed
>> module is being deleted and de-indexed.
>
>
> Excellent suggestion!  I think modules@ is the right list and suspect the
> frequency is rare.  (Maybe Andreas has statistics on it?)
>
> Could you please open a PAUSE ticket for that idea?
>
>
> --
> David Golden  Twitter/IRC/Github: @xdg
>

Re: Thoughts on Kik and NPM and implications for CPAN

[Karen Etheridge]

Should an author be able to delete a currently-indexed distribution on the
CPAN? Yes, without reservation or exception. Open source is free, and that
freedom includes removing my consent for my name to be attached to a
publication at any time.

However, we (the CPAN community) can do a lot of things after that to
mitigate any damage. I wholeheartedly agree with transferring namespace
permissions to something that the PAUSE admins control, so any random joe
cannot claim the namespace and upload whatever he likes into it (this is an
attack vector we must keep closed).  We also need to be able to act quickly
to publish something in its place so installations pulling directly from
the CPAN do not break.  I would suggest an email alert go out to the
modules@ list (or another list, should this prove too noisy) providing
notification that an indexed module is being deleted and de-indexed.

Re: Thoughts on Kik and NPM and implications for CPAN

[Todd Rinaldo]

> On Mar 23, 2016, at 11:20 AM, David Golden  wrote:
> 
> On Wed, Mar 23, 2016 at 11:25 AM, Stefan Seifert  > wrote:
> > * I think we have to allow mass deletion, even if that de-indexes stuff.  I
> > think that's an author's right.
> 
> I've never gotten that argument.
> 
> Let's try a narrower argument: Should an author be allowed to delete *any* 
> distribution?
> 
> * Old tarballs?  Seems reasonable.
> 
> * Currently indexed tarballs?  What if a file was included that was never 
> meant for publication?  What if there was a really dangerous bug?  What if it 
> was accidentally uploaded company code that *isn't* open source?
> 
> I can think of several legitimate reasons to allow deletion and de-indexing.
> 
> Moreover, if we disallowed deletion, an author could just upload an empty 
> module except for a higher version number and get that indexed and that is as 
> effective at breaking things as removal.
> 
> So given that removal (a) has several reasonable uses and (b) doesn't stop 
> authors from mass-breaking dependents if they want to, I see no reason to 
> prohibit it.
> 
> David

I agree with you taking away delete doesn't solve anything. So at best, all we 
can do is mitigate the catastrophes when they happen.

For me the scenario I worry about is: KWILLIAMS declares Module::Build a 
failure. He then removes all co-maints and wipes all of his tarballs. IMO, 
PAUSE admins should have a right to say: NOPE. Leon is now the owner of M::B, 
especially if the module removal breaks a large enough part of CPAN.

+1 to addressing https://github.com/andk/pause/issues/169 
. This is a potential security issue 
waiting to happen.

Todd

Re: Thoughts on Kik and NPM and implications for CPAN

[David Golden]

On Wed, Mar 23, 2016 at 11:25 AM, Stefan Seifert 
wrote:

> > * I think we have to allow mass deletion, even if that de-indexes
> stuff.  I
> > think that's an author's right.
>
> I've never gotten that argument.


Let's try a narrower argument: Should an author be allowed to delete *any*
distribution?

* Old tarballs?  Seems reasonable.

* Currently indexed tarballs?  What if a file was included that was never
meant for publication?  What if there was a really dangerous bug?  What if
it was accidentally uploaded company code that *isn't* open source?

I can think of several legitimate reasons to allow deletion and de-indexing.

Moreover, if we disallowed deletion, an author could just upload an empty
module except for a higher version number and get that indexed and that is
as effective at breaking things as removal.

So given that removal (a) has several reasonable uses and (b) doesn't stop
authors from mass-breaking dependents if they want to, I see no reason to
prohibit it.

David


-- 
David Golden  Twitter/IRC/Github: @xdg

Re: Thoughts on Kik and NPM and implications for CPAN

[Sawyer X]

Related to this perhaps was the Ion3 debacle:

https://en.wikipedia.org/wiki/Ion_%28window_manager%29#Controversy

Long story short: Ion3 developer did not want a certain feature.
Debian added a patch for it. He got mad, pulled Ion3 out. Same with
ArchLinux, NetBSD, and FreeBSD.



On Wed, Mar 23, 2016 at 4:25 PM, Stefan Seifert  wrote:
> On Wednesday 23 March 2016 11:07:34 David Golden wrote:
>
>> * I think we have to allow mass deletion, even if that de-indexes stuff.  I
>> think that's an author's right.
>
> I've never gotten that argument. The code in question is usually under a very
> permissive license. Publishing code under such a license is a very conscious
> decision of the author. People trust the author and build on this foundation.
> Among those people are the ones that run CPAN and its mirrors. They too are
> only allowed to distribute the code because the license says so. When people
> download distros from CPAN they do so as sub licensees of whoever runs their
> favorite CPAN mirror.
>
> Now if the original author decides to no longer publish her code, that's
> absolutely fine. I just don't get why CPAN should follow suite and do the
> same. We don't demand this of BackPAN and we don't demand the same from other
> users who trusted the license. Why is CPAN literally the only entity that
> should go beyond the license and do the author's bidding? Considering that
> copyright exists solely to benefit the public, I have to ask: how is the
> public served by this self censorship?
>
> Stefan

Re: Thoughts on Kik and NPM and implications for CPAN

[Stefan Seifert]

On Wednesday 23 March 2016 11:07:34 David Golden wrote:

> * I think we have to allow mass deletion, even if that de-indexes stuff.  I
> think that's an author's right.

I've never gotten that argument. The code in question is usually under a very 
permissive license. Publishing code under such a license is a very conscious 
decision of the author. People trust the author and build on this foundation. 
Among those people are the ones that run CPAN and its mirrors. They too are 
only allowed to distribute the code because the license says so. When people 
download distros from CPAN they do so as sub licensees of whoever runs their 
favorite CPAN mirror.

Now if the original author decides to no longer publish her code, that's 
absolutely fine. I just don't get why CPAN should follow suite and do the 
same. We don't demand this of BackPAN and we don't demand the same from other 
users who trusted the license. Why is CPAN literally the only entity that 
should go beyond the license and do the author's bidding? Considering that 
copyright exists solely to benefit the public, I have to ask: how is the 
public served by this self censorship?

Stefan

Re: Thoughts on Kik and NPM and implications for CPAN

[Sawyer X]

Well thought-out. I agree.
(I'd add more but really, there's no need. :)

On Wed, Mar 23, 2016 at 4:07 PM, David Golden  wrote:
> If you don't know what I'm referring to, read
> http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
>
> Leaving aside the IP issue, I think it might be worth considering what would
> currently happen if someone chose a 'mass removal' and whether that's what
> we'd like to have happen.
>
> N.B. this is more extreme than
> http://www.xenoterracide.com/2015/05/abandoning-all-perl-modules.html --
> that dropped perms, but left the tarballs indexed.  What if someone goes
> beyond that...
>
> Consider a scenario for user "Pat":
> * Pat schedules all tarballs for deletion and waits 3 days
> * All tarballs are deleted by PAUSE
> * mldistwatch de-indexes any previously indexed tarballs
> * Pat removes all comaints for all modules
> * Pat drops primary permissions on all modules
> * Pat drops co-maint perms on all modules
>
> At that point, anything depending on Pat's tarballs is broken, as they
> aren't indexed (ignoring for the moment cpanm's use of backpan indexes).
>
> Also, I think the next tarball uploaded with a namespace previously
> controlled by Pat gets "first come" permissions and is indexed (regardless
> of version number).
>
> Have I got that scenario right?
>
> My thoughts:
>
> * I think we have to allow mass deletion, even if that de-indexes stuff.  I
> think that's an author's right.
>
> * I think we should *not* free up namespaces for random takeover
>
> * I think PAUSE admins should consider a reasonable request by a
> responsible-seeming party to take over a namespace (e.g. by forking a
> tarball from BackPAN).
>
> In other words: authors own their tarballs, but PAUSE owns the namespaces
> (and periodically delegates responsibility to a maintainer).
>
> Mechanically, I think that means that when PAUSE is dropping permissions, it
> should instead transfer control to a PAUSE-controlled ID.  (Effectively,
> https://github.com/andk/pause/issues/169 )
>
> Thoughts?
>
> David
>
> --
> David Golden  Twitter/IRC/Github: @xdg