I'd rather like to avoid adding additional complexity to the current
index files.
Instead, what about switching to a native SQLite index for the new system.
This is working quite nicely in the JSAN now, and it greatly
simplifies things because you can download the index much more simply.
1. Pull gzip
2. Expand
3. Connect
If we wanted to fragment the index, we could just download the extra
parts of the index and then do an ATTACH DATABASE to load in the
additional fragment.
This would, of course, mean that SQLite and DBI support is needed in
the "core" for Perl 6.
But since (I gather) the P6 core will be even lighter than the Perl 5
one, and there will be a sort of "Standard Library" defined
independantly that includes the modules needed to install other
modules, this might not be too hard to overcome.
How hard would it be, in principle, to have SQLite and DBI support in
Perl 6? Can you do all the necessary C fu yet?
Failing this, a simple idea has been to produce a JSON structure which
has a similar logical structure to the database.
{
author_table: ....
...
}
Adam K
On Tue, Jan 5, 2010 at 9:15 AM, brian d foy <brian.d....@gmail.com> wrote:
> In article <20100104154532.ge6...@bestpractical.com>, Jesse Vincent
> <je...@bestpractical.com> wrote:
>
>
>> This brings us to "How do we get there from here?" Who are the folks
>> working on the Perl6 CPAN client?
>
> Well, here's what I'm doing:
>
> I think the first step is very simple. My interest is porting the Perl
> 6 module ideas into Perl 5. However, the code or implemention don't
> matter. Once we know the answer, it's easy to go from whatever I might
> do in Perl 5 to something in Perl 6.
>
> 02packages is actually quite flexible. There's a line in the header
> that tells consumers what the columns are. Since we've used the same
> three columns, a lot of stuff has assumed that the order and meaning
> will always be the same. We fix the current stuff to pay attention to
> the headers. If we reorder the columns or stick in a new column,
> nothing breaks.
>
> After that, we produce additional indices that include more
> information, such as the author, a "canonical" flag perhaps, and other
> stuff. This isn't necessarily something that PAUSE would do. Alternate
> authorities can make indices. For the past year I've been working with
> a couple of paying clients to make custom CPAN repositories. I have
> most of the pieces in place. Think, for instance, about a market
> segment
> that wants to only use CPAN as it was on June 1, when their application
> went into code freeze. I can produce an index that replicates that.
> That's not the same thing as saving the PAUSE index since dists
> disappear arbitrarily.
>
> Once we have the index, we have a way to ask for what's available, say,
> for CGI.pm. That's easy, in the sense that you don't have to draw too
> much on a whiteboard, and not so bad in the pause and CPAN.pm code for
> those of us who have our noses in there constantly.
>
> Okay, all of that is step one and it's something I'm working on.
>
> Step two is figuring out how to decide which all of the things that are
> available are the most suitable. That's where a lot of people argue
> endlessy in mailing lists, then Andreas or Jarkko or somebody tells us
> the answer.
>
