Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Tim Bunce
Currently on PAUSE you have to explicitly delete old uploads. How about changing it so you have to explicitly KEEP old uploads that appear to have been superseded? PAUSE already has a mechanism to delete files at some future point in time. That's currently only used as part of a safety/sanity che

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread David Golden
On Thu, Mar 25, 2010 at 7:12 AM, Tim Bunce wrote: > Currently on PAUSE you have to explicitly delete old uploads. > > How about changing it so you have to explicitly KEEP old uploads > that appear to have been superseded? I would support an option to purge automatically only if the default is NOT

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ruslan Zakirov
Hi, I think David is not alone here. As PAUSE has interfaces for users to delete then it's possible to write a cpanpurge utility to do it without changing PAUSE. Small yaml config in home dir with protections of particular files and interactive mode. On Thu, Mar 25, 2010 at 2:52 PM, David Golden

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ovid
--- On Thu, 25/3/10, David Golden wrote: > From: David Golden > I don't think it's a good idea to make it hard for people > to find > older versions of a distribution -- where hard means "have > to track it > down on backpan".  (Though we could make clients > better about it, I > supposed.) I

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread David Cantrell
On Thu, Mar 25, 2010 at 11:12:32AM +, Tim Bunce wrote: > How about changing it so you have to explicitly KEEP old uploads > that appear to have been superseded? Why? Is there a problem with the size of the CPAN? Have any mirror maintainers complained? -- David Cantrell | even more awesome

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Barbie
On Thu, Mar 25, 2010 at 11:12:32AM +, Tim Bunce wrote: > Currently on PAUSE you have to explicitly delete old uploads. Which often is a good thing. While BACKPAN exists, it isn't somewhere that many go to look for old distributions. For me and probably others, BACKPAN only distributions are on

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Graham Barr
On Mar 25, 2010, at 8:42 AM, Barbie wrote: > > Lastly I would also personnally be annoyed if only the latest versions > were available, as I often make great use of the diff tool on > search.cpan.org. Having only the latest version renders that great tool > redundant :( I use that too :-) and it

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread David Cantrell
On Thu, Mar 25, 2010 at 01:42:58PM +, Barbie wrote: > There are many distributions on CPAN that older versions work on a > particular perl/os, but more recent ones don't. Latest isn't necessarily > the greatest. > > If you are going to perform this then it should really feed off the CPAN > Te

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Jarkko Hietaniemi
I have one case where the v1 and v2 of a module are simply incompatible, but v1 still works, and unless the users have a compelling reason, they won't migrate. Pulling the rug from under them would be quite unsportsmanlike. Deletion should be opt-in, and there should be a way to "pin" some releas

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ask Bjørn Hansen
On Mar 25, 2010, at 4:12, Tim Bunce wrote: > Currently on PAUSE you have to explicitly delete old uploads. > > How about changing it so you have to explicitly KEEP old uploads > that appear to have been superseded? I like it. I agree with Jarkko that there should be a way to "pin" some version

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Chris Nandor
What Jarkko said. On Mar 25, 2010, at 08:00, Jarkko Hietaniemi wrote: > I have one case where the v1 and v2 of a module are simply > incompatible, but v1 still works, and unless the users have a > compelling reason, they won't migrate. Pulling the rug from under > them would be quite unsportsman

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Chris Nandor
On Mar 25, 2010, at 08:10, Ask Bjørn Hansen wrote: > I agree with Jarkko that there should be a way to "pin" some versions and the > configuration should be "more than N newer releases" or some such. > > I think it should be on by default though. Older than 3 (or 6?) months and > at least 2 or

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Andy Armstrong
On 25 Mar 2010, at 15:36, Chris Nandor wrote: > I like that solution better [snip] But solution to what? Are we convinced there's actually a problem here? -- Andy Armstrong, Hexten

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ricardo Signes
* Andy Armstrong [2010-03-25T11:38:46] > On 25 Mar 2010, at 15:36, Chris Nandor wrote: > > I like that solution better > > But solution to what? Are we convinced there's actually a problem here? I am entirely unconvinced. -- rjbs

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ask Bjørn Hansen
On Mar 25, 2010, at 8:38, Andy Armstrong wrote: >> I like that solution better > > > [snip] > > But solution to what? Are we convinced there's actually a problem here? CPAN has almost 200k files. www.cpan.org says there are "17627 modules". rsyncing a gazillion files doesn't work that well

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Andy Lester
On Mar 25, 2010, at 10:38 AM, Andy Armstrong wrote: > But solution to what? Are we convinced there's actually a problem here? The first two rules of optimization club: 1) You do not optimize. 2) You do not optimize without measuring. As soon as someone can explain specifics of the problem, inc

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Eric Wilhelm
# from Ask Bjørn Hansen # on Thursday 25 March 2010 08:55: >> But solution to what? Are we convinced there's actually a problem >> here? > >CPAN has almost 200k files.  www.cpan.org says there are "17627 > modules".  rsyncing a gazillion files doesn't work that well (on the > server).  Helping aut

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Eric Wilhelm
# from David Golden # on Thursday 25 March 2010 04:52: >Of all the things  I'd  like to do with PAUSE, this feature is very, >very low on my list. Make it easier to create a "mock pause" perhaps? Then at least someone could demo their suggested feature and maybe it would be easier to drop-in t

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread nadim khemir
> > On Mar 25, 2010, at 8:38, Andy Armstrong wrote: > > >> I like that solution better > > > > > > [snip] > > > > But solution to what? Are we convinced there's actually a problem here? > > CPAN has almost 200k files. www.cpan.org says there are "17627 modules". rsyncing a gazillion fil

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Geoffrey Broadwell
On Thu, 2010-03-25 at 13:23 -0700, Eric Wilhelm wrote: > # from Ask Bjørn Hansen > # on Thursday 25 March 2010 08:55: > > >> But solution to what? Are we convinced there's actually a problem > >> here? > > > >CPAN has almost 200k files. www.cpan.org says there are "17627 > > modules". rsyncing a

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Barbie
On Thu, Mar 25, 2010 at 02:08:45PM -0700, Geoffrey Broadwell wrote: > > Forgive a lurker, but wasn't that the point of this: > > http://search.cpan.org/~andk/File-Rsync-Mirror-Recent-0.0.7/ > > When I saw that announced, I remember thinking "Yay, large archive rsync > problem solved!" Did i

Re: mirroring the CPAN more efficiently

2010-03-25 Thread Eric Wilhelm
# from Geoffrey Broadwell # on Thursday 25 March 2010 14:08: >> Maybe CPAN mirrors are more easily updated than via a generic rsync? >>  Is the burden only network/cpu for checking whether a bunch of old >> archives have changed, or does disk matter? > >Forgive a lurker, but wasn't that the point

Re: mirroring the CPAN more efficiently

2010-03-25 Thread David Golden
It's a real memory hog. I don't think it needs to be, but haven't had the tuits to prove that assertion. David On Mar 25, 2010 6:43 PM, "Eric Wilhelm" wrote: # from Geoffrey Broadwell # on Thursday 25 March 2010 14:08: >> Maybe CPAN mirrors are more easily updated than via a generic rsync? >>

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ask Bjørn Hansen
On Mar 25, 2010, at 14:08, Geoffrey Broadwell wrote: >> Maybe CPAN mirrors are more easily updated than via a generic rsync? Is >> the burden only network/cpu for checking whether a bunch of old >> archives have changed, or does disk matter? > > Forgive a lurker, but wasn't that the point of

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Ask Bjørn Hansen
On Mar 25, 2010, at 13:23, Eric Wilhelm wrote: > Maybe CPAN mirrors are more easily updated than via a generic rsync? Is > the burden only network/cpu for checking whether a bunch of old > archives have changed, or does disk matter? Most CPAN mirrors use rsync. It's not realistic to make the

Re: rsync and the CPAN

2010-03-25 Thread Eric Wilhelm
# from Ask Bjørn Hansen # on Thursday 25 March 2010 16:55: >Most CPAN mirrors use rsync.  It's not realistic to make them change > that ("Hello all mirror operators -- so that tool that you use for > ALL YOUR MIRRORS; well ... maybe you can use something else for > us?"). > >rsync is all disk i/o

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Adam Kennedy
What he said. Most people don't mirror CPAN. They mirror many things. This is the same reason we've struggled with statistics. How do you ask someone mirroring three dozen different things to put in a special log-munging tool just for us. Adam K On Fri, Mar 26, 2010 at 10:55 AM, Ask Bjørn Hanse

Re: Trimming the CPAN - "Automatic Purging"

2010-03-25 Thread Geoffrey Broadwell
On Fri, 2010-03-26 at 15:36 +1100, Adam Kennedy wrote: > What he said. > > Most people don't mirror CPAN. They mirror many things. > > This is the same reason we've struggled with statistics. How do you > ask someone mirroring three dozen different things to put in a special > log-munging tool ju