Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-26 Thread Ryan Schmidt
On Feb 25, 2008, at 15:34, Rainer Müller wrote: > Jordan K. Hubbard wrote: > >> Because given the $portname-$portversion naming of 99% of the >> distfiles (and the unique names of the 1% that are left), I just >> don't >> see collisions as a problem. I don't see it as a problem with the >> Free

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-25 Thread Landon Fuller
On Feb 25, 2008, at 1:34 PM, Rainer Müller wrote: I didn't know they are using such a flat namespace. They're not -- When there are distfile collisions (and there are), they use DIST_SUBDIR to partition a particular port's distfiles. MacPorts by default simply places all distfiles in a por

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-25 Thread Rainer Müller
Jordan K. Hubbard wrote: > Because given the $portname-$portversion naming of 99% of the > distfiles (and the unique names of the 1% that are left), I just don't > see collisions as a problem. I don't see it as a problem with the > FreeBSD ports collection either, and they're almost 3X bigge

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-25 Thread Jordan K. Hubbard
On Feb 24, 2008, at 9:29 PM, Rainer Müller wrote: > Jordan K. Hubbard wrote: >>> All distfiles in one single directory? I am against that at all! >> Why? Collisions? If so, please name the collisions in question, >> because I cannot find any. > > Maybe not yet, but maybe they will come in l

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread Anders F Björklund
Jordan K. Hubbard wrote: > I fail to understand this stubborn insistence on date stamp and group > information for files which are simply not meant to be user visible. > There is no group information you need to see on the distfile mirror > because you won't be managing the distfile mirror. I wor

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread Rainer Müller
Jordan K. Hubbard wrote: >> All distfiles in one single directory? I am against that at all! > > Why? Collisions? If so, please name the collisions in question, > because I cannot find any. Maybe not yet, but maybe they will come in later? Why not be collisions aware? > If you add indirect

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread William Siegrist
It would match the layout of the ports in the svn repo and make the tree a little more manageable for human consumption. I could see providing Apache indexes for the tree so people could browser and grab distfiles manually. But mainly my suggestion was based on matching svn. -Bill On Fe

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread Jordan K. Hubbard
On Feb 24, 2008, at 7:46 PM, Rainer Müller wrote: > Jordan K. Hubbard wrote: >> On Feb 23, 2008, at 8:05 AM, William Siegrist wrote: >>> So whats left? I think you'll want a special hostname for these? >>> Maybe http://distfiles.macports.org///? >> I'm not sure what value adds. / would seem

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread Rainer Müller
Jordan K. Hubbard wrote: > > On Feb 23, 2008, at 8:05 AM, William Siegrist wrote: > >> So whats left? I think you'll want a special hostname for these? >> Maybe http://distfiles.macports.org///? > > I'm not sure what value adds. / would seem more > than reasonable, and a completely flat name

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread Jordan K. Hubbard
On Feb 23, 2008, at 8:05 AM, William Siegrist wrote: So whats left? I think you'll want a special hostname for these? Maybe http://distfiles.macports.org///? I'm not sure what value adds. / would seem more than reasonable, and a completely flat namespace even more so since it then beco

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-24 Thread Anders F Björklund
Jordan K. Hubbard wrote: >> Before fetching all the files, it would probably nice to fix the >> timestamps so that they correctly reflect the upstream distfile ? > > I guess I'm missing something. Why is this important? Distfiles > are essentially opaque to the MacPorts user, so what would be

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-23 Thread Jordan K. Hubbard
On Feb 22, 2008, at 11:51 PM, Anders F Björklund wrote: > Before fetching all the files, it would probably nice to fix the > timestamps so that they correctly reflect the upstream distfile ? I guess I'm missing something. Why is this important? Distfiles are essentially opaque to the MacPor

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-23 Thread Ryan Schmidt
On Feb 23, 2008, at 10:09, William Siegrist wrote: > On Feb 22, 2008, at 11:51 PM, Anders F Björklund wrote: > >> Before fetching all the files, it would probably nice to fix the >> timestamps so that they correctly reflect the upstream distfile ? >> >> http://trac.macports.org/projects/macports/t

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-23 Thread William Siegrist
The servers are running Tiger, but have curl installed via MP, so this shouldnt be a problem unless /usr/bin is hardcoded for this? Or am I missing the issue? -Bill On Feb 22, 2008, at 11:51 PM, Anders F Björklund wrote: Jordan K. Hubbard wrote: On Feb 22, 2008, at 7:11 PM, Rainer Mülle

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-23 Thread William Siegrist
I didnt mean to add any unnecessary work for MP. My main point was that per-commit versus daily is the same work for me. I dont have any hard stats, but I suppose most Portfile changes result in some sort of distfile change (most are version bumps?), so we can probably just make it work wit

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread Anders F Björklund
Jordan K. Hubbard wrote: > On Feb 22, 2008, at 7:11 PM, Rainer Müller wrote: > >> One of these things would be how we would push files on the mirror? >> If we keep master_sites we could use something like `port fetch all' >> in a cronjob or in a post-commit hook. But that will have the >> problem

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread Jordan K. Hubbard
On Feb 22, 2008, at 9:34 PM, William Siegrist wrote: > We dont really see any dips in server load, so waiting until night > (or any time in particular) doesnt buy us anything. I'm assuming > you still agree with only fetching distfiles when they change, and > we already have all the work d

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread William Siegrist
On Feb 22, 2008, at 9:02 PM, Jordan K. Hubbard wrote: On Feb 22, 2008, at 8:08 PM, William Siegrist wrote: The server would grab distfiles during post-commit I imagine (assuming the checksums get changed, patchfiles added, etc). So similar to the way we handle linting of Portfiles during p

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread Jordan K. Hubbard
On Feb 22, 2008, at 8:08 PM, William Siegrist wrote: > The server would grab distfiles during post-commit I imagine > (assuming the checksums get changed, patchfiles added, etc). So > similar to the way we handle linting of Portfiles during post- > commit, we would check for these changes.

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread Jordan K. Hubbard
On Feb 22, 2008, at 7:11 PM, Rainer Müller wrote: > One of these things would be how we would push files on the mirror? > If we keep master_sites we could use something like `port fetch all' > in a cronjob or in a post-commit hook. But that will have the > problem that it currently port fetc

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread William Siegrist
The server would grab distfiles during post-commit I imagine (assuming the checksums get changed, patchfiles added, etc). So similar to the way we handle linting of Portfiles during post-commit, we would check for these changes. I just dont want to implement some hack of parsing Portfiles m

Re: MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

2008-02-22 Thread Rainer Müller
William Siegrist wrote: > I already made the offer to portmgr a while back, so they know. I > think there needs to be some added API to macports in order to make > the engineering a little cleaner server-side, but there hasnt been > much discussion yet. So if anyone wants to take the lead o