based on the detail earlier in the thread, this seems to make sense. the question i'm left with is what if there's a custom sort order for this project? maybe something that uses unicode characters, or right-to-left sort order, or ....
i don't have a concrete example (it would really make thinking about this easier) but i wonder if there should be a way to specify a custom sort sub that can be run as a callback. it doesn't make sense to put that in each release, though, since this is really a project-level concept. so, is there a place to put a custom sort sub per project, or will it have to be baked in to deploy-config? ~jerry -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Phillip Moore Sent: Thursday, 12 January, 2012 16:31 To: EFS core development list Subject: Re: [EFS-dev] Project sorting algorithms Better idea -- write this into the metadata.conf file for each release. [config] sort = gnu | perl | text | whatever It's in the efsdeploy config rules, so just write it out when the releases are installed. We're already parsing the metadata.conf files in EPD, so this information will be readily available. This also means I can get this implemented very simply, and I should have a solution my the middle of next week. On Thu, Jan 12, 2012 at 5:45 PM, Phillip Moore <[email protected]> wrote: > OK, I've made some progress here, but the results are not very good. > > I tested out a patch to efs-perl that treats versions as v-strings, > but that's just a trade off. That algorithm can sort HTML-Templates > versions, but then it breaks on Module-Build and DateTime. There > doesn't not exist a single algorithm that correctly implement version > sorting automatically. I am convinced we have no choice here but to > manage this with project-specific metadata. > > Now, as noted below, it's easy to manage this in efsdeploy's config > data, however that information is intended to be a BUILD-time > dependency ONLY. I do not, under any circumstances, want that data > to have RUNTIME-dependencies. > > The only realistic solution I can see is to have efs-perl read this > information from something like: > > /efs/dist/perl5/EFS-Perl-Config/current > > IOW, we'll still use an autorelease (the EFS 3 replacement for incr > releases -- they kick ass), but it will be compiled from the > deploy-config data, and distributed separately. In reality, we will > only need to specify a special sort order as an exception in the perl5 > metaproj, so I expect this data to change fairly infrequently. The > deploy-config stuff, OTOH, will change a LOT, and this will allow us > to update the deploy-config stuff as often as we want, with no risk to > your production systems. > > This seems reasonable for several reasons. First, remember the edge > condition this applies to. When EPD is expanding a dependency tree, > and there are two DIFFERENT releases specified at the same level of > the dependency tree, then and only then does this code kick in. > Honestly, we're splitting hairs here, but it's important to get this > right. > > I'm working through a rebuild of my perl5/core release, to introduce > the perl5/EFS-Perl-Indirect project, and this is a good time to code > something up for EFS-Perl-Config, which just be a couple of hooks in > some efsdeploy rules. It should be straight forward to have efs-perl > query this information. > > On Mon, Jan 9, 2012 at 12:03 PM, Phillip Moore > <[email protected]> wrote: >> I'm in the middle of totally reworking how dependencies are managed >> in the efsdeploy code, and I've finally run into a problem I've been >> avoiding for a while. How do we manage the metadata that specifies >> how a project's releases are sorted? >> >> This is a LOT harder than you might think. First of all, the main >> problem is that we need this chunk of metadata in several places. >> The first time this problem reared it's head was EFS::Perl::Depends, >> which was really the first time we had to think about this. The >> first major release of EPD got it totally wrong, and used >> Sort::Versions to sort everything, which is really only correct for a >> subset of the CPAN distros. In V2 I fixed this by switching to >> version objects, which is the correct default, but then you discover >> that there are distros like HTML-Template that use versions which do >> NOT sort correctly with version.pm. Those, you have to use Sort::Versions >> for. >> >> This defines an important requirement for managing this information: >> it has to be available to EPD, and be CHEAP. EFS database >> attributes would be a nice place to manage this, but we can't have >> sitecustomize.pl querying a database for every perl invocation. >> Right now, EPD is dependent only on the metadata.conf text files >> found in each release. Wherever we end up managing this information, >> it will have to be written out into these metadata.conf files by >> efsdeploy (easy). >> >> RIght now, I see 3 distinct sorting algorithms. Here's the keywords >> I'll be using in the code to describe them: >> >> [*] gnu (Sort::Versions) >> >> Every single project on gnu.org uses a versioning scheme that sorts >> correctly with Sort::Versions, and this will probably work for most >> open source projects that use a major.minor.subminor release >> convention. >> >> [*] perl (version objects) >> >> MOST of CPAN can be sorted using version.pm objects, with a few >> exceptions we'll have to configure, as described above. >> >> [*] text (natural perl "sort" order) >> >> This is how you would sort, say, EFS autoreleases (YYYYMMDD-HHMMSS), >> for example, or anything that uses a convention that can be sorted as >> plain text. Autoreleases are a stupid example, though -- you always >> access them (BY DESIGN) through the current releasealias. >> >> So, where do we manage this data? We want to be able to specify it >> in ONE place, and then have it propogate wherever else it is needed >> as a by-product of the automated efsdeploy workflow. Seems obvious >> to me this project-specific information that should be in the >> deploy-config rules, therefore in globals.conf: >> >> [config] >> sort = gnu | perl | text >> >> This will allow us to specify system defaults, so the gnu system >> would obviously default "gnu", perl5 would default to "perl". >> >> This would make things easy for efsdeploy, which is doing the >> expansions per-project. However, for EFS::Perl::Depend (the only >> other consumer of this information right now), it will be a little >> trickier, since it currently just assumes "perl" and falls back on >> "gnu" (i.e. using the Sort::Versions algorithm) only when the >> releases can't be converted into version objects. >> >> There are two approaches I could take with EPD. First, since we now >> (as of last week) have all the project-specific efsdeploy rules >> available in the various */deploy-config-* projects, it might be >> possible for EPD to query that data. The second approach would be >> to only worry about this when we have to. For example, if a project >> only has one release, then there's nothing to sort. Only if a >> project has two or more releases do we need to query the sort >> algorithm. In this case, we could read the metadata.conf for all >> the releases, make sure they are the same and then sort. if the >> releases support different algorithms, abort, and require the >> dependency to be specified manually. >> >> Now, with either solution, we're looking at more filesystem overhead. >> I'm leaning towards this algorithm: >> >> If project-specific efsdeploy rules exist, then use that sort >> algorithm If not, use the current algorithm (try version.pm, fall >> back on S::V) >> >> I think the IO overhead, as long as we memoize the values for each >> project, should prove minimal for EPD. >> >> If anyone's watching, you'll see some commits over the next few days >> that implement all of this. _______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev ---------------------------------------------------------------------- This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing. _______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev
