Re: Package choosing algorithm ... more clarification
For those interested, see attached file (chooser_overview.txt). Regards, Henrichooser_overview.txt: 2016-04-05 // Another attempt to create an overview ... = precondition - assume a "database" which reflects the info from setup.ini ... (packagedb::packages) - it list each package and every version of that pkg ... - this database even tells us what version has been installed if that is the case (reflecting installed.db) - furthermore we (setup.exe) are able to ascertain whether the tarball, associated with a version, is available or not -- the source code of setup.exe uses the term "accessible" for availability = D w/o I (Download without Installing) - setup.exe should show us ALL the pkgs and every version of that pkg ... - assume all tarballs are available (stable mirror) = I f LD (Install from Local Directory) - setup.exe should show us the installed pkgs and the versions of that pkg that are not installed, but are available in the repo ... - the tarball of an installed version may or may not be available in the repo ... - furthermore it should show us each non-installed pkg if at least one version of that pkg is available in the repo ChooserPage::OnInit has the following call flow in both cases (D w/o I and I f LD) ChooserPage::OnInit packagemeta::ScanDownLoadedFiles packagedb::setExistence packagemeta::trustp(true, ... = D w/o I // (pkg.installed == version) => pkg is installed - ScanDownloadedFiles # method name is a misnomer here ! (pkg is installed || a version is acccessible) => erase version from packagemeta.versions - here all versions will be kept (stable mirror) in packagemeta.versions - setExistence ! (pkg is installed || a version is acccessible) => erase pkg from packages - the test whether the pkg is installed or not, is useless here ... - ... as all pkgs (i.e. all versions of each pkg) are available (accessible): UNmodified database End result: setup.up will show us all the pkgs and every version of each pkg ... = I f LD // (pkg.installed == version) => pkg is installed - ScanDownloadedFiles ! (pkg is installed || a version is accessible) => erase version from packagemeta.versions - not-installed pkgs: available versions will be kept in packagemeta.versions - installed pkgs: the installed version will be kept - installed pkgs: non-available versions will be removed from packagemeta.versions - setExistence ! (pkg is installed || a version is accessible) => erase pkg from packages - a not-installed pkg, of which at least one version is available: this pkg will be kept in the database (versions will show the available versions) - installed pkgs: these pkgs will be kept in the database (versions will show the installed version and available versions that are not installed) End result: setup.exe will show us all the available versions of each pkg, if it has not been installed yet, and all the pkgs (both the installed version and available versions that are not installed) which have been installed ... - Ignoring the issue of source tarballs ... Each "package entry" (p, c and/or t) in setup.ini is represeneted by an object of type packagemeta ... - class variable packages (of type packagedb): std::map or: a mapping from a packagename to a pointer to an object of type packagemeta A simplified view of the data members of class packagemeta is as follows: - prev of type packageversion - curr ditto - exp ditto - installed ditto - versions of type std::set - prev, curr, exp and installed reflect the "static" part of a "package entry": these data members reflect the info from both setup.ini and installed.db ... "Nullifying" these data members, as currently happens in ScanDownLoadedFiles is ... stupid? versions tells what versions of the pkg are available (accessible); versions is managed by ScanDownLoadedFiles ... Attempting to grasp the general flow of computation: packagemeta::ScanDownLoadedFiles => for each pkg // from packagedb::packages for each version // from metapackage::versions [ .scan() ] // packageversion::scan() -- simplified! // (pkg.installed == version) => pkg is installed if ! (pkg is installed || version is accessible) // packagemeta::installed, packageversion::accessible() erase version from versions Note: a version is NOT erased if the version is installed OR if the version is available packagemeta::accessible => // called by setExistence for each version // from metapackage::versions if version is accessible // packageversion::accessible() return true // pkg is available in (local) repo return false packageversion::accessible => _packageversion::accessible => ... packagedb::setExistence for each pkg // from packagedb::packages // (pkg.installed == version) => pkg is installed if ! (pkg is installed || a version is accessible) // packagemeta::installed, packagemeta::accessible() er
Re: Package choosing algorithm ...
Title: Re: Package choosing agorithm ... Hi Corinna, (Of course I am formally addressing the project leader, but Achim, Jon, Yaakov, others? are welcome to correct me if I am wrong below). D w/o I = Download without Install I f LD = Install from Local Directory As a reminder: I reported that a new test release of Cygwin is NOT offered for install (having installed the previous test release) in my case. https://cygwin.com/ml/cygwin/2016-03/msg00425.html Relevant in my case is: - I am not using a local mirror ... - but a "partially populated Local Directory" - I do not invoke setup.exe with the -m option - my local directory does not have the current release of Cygwin As you replied "it works for me" and "not using the -m option should not make a difference", I decided to study the source code of setup.exe. As I had never seen the source code of setup before, I started my study where you had "left off" modifying (about a year ago). Subsequently, I reported back to the list, announcing that I found the bug, but was not sure I had. https://cygwin.com/ml/cygwin/2016-03/msg00480.html I was wrong. My patch did work, but for the wrong reasons. So does your modification of packagemeta::trustp ... Here is another attempt of mine (attempt? I seriously doubt whether there is someone which fully understands the source code in all its details ...) You will find 2 patch files attached. These patch files modify (correct): 1. packagemeta::ScanDownLoadedFiles and 2. packagemeta::trustp (yes, I do realize, that this correction to setup, is NOT important enough to most of you - using a local mirror and all that; consequently, do whatever you like with my post - I am merely providing info to the list). My argument for the correction is as follows: - packagedb::packages, a class variable, is the "package database" that is used by setup. - basically, it is a list of pairs (mapping between the packagename and its entry in setup.ini) (one where installed.db has been "integrated"). - formally: - typedef std::map packagecollection; - static packagecollection packages; - packagemeta is the type that describes a "package entry" from setup.ini - an object of type packagemeta does not only describe the previous, current and experimental (if applicable) lines from a package entry in setup.ini, it also describes the installed version of a package, as if installed.db is part of setup.ini. -- - bottom-line: the prev, curr, exp and installed data member of packagemeta are "static": these data members represent the info from setup.ini (and installed.db) and they should never be "nullified". - yet, that is exactly what happens in metapackage::ScanDownLoadedFiles. -- - packagemeta ALSO has a "not so static" data member, called versions; like packages, the contents of versions is "variable" - versions, a set of type packageversion, contains the versions of the package that are available (the source code speaks about "accessible") - available (in my simplistic view) means whether or not the tarball is present in the (local) repo - unfortunately, though versions specifies which versions are available, it does not tell us whether the previous, current or experimental version of a package is available or not -- - bottom-line: version numbers can be compared if that info is present in setup.ini; however, although the info may be present, it does not tell us that the associated tarball is available (neither does versions). - consequently, metapackage:trustp does not only have to verify that setup.ini has the required info, it also has to verify that the tarball is available, before it can give the "go-ahead". Lastly: No, I have not tested (been able) my modification for each possible configuration out there (for instance, I do not install source tarballs). Regards, Henri =diff --git a/package_meta.cc b/package_meta.cc index 34ff78c..ff787a8 100644 --- a/package_meta.cc +++ b/../src2/package_meta.cc @@ -683,15 +683,18 @@ packagemeta::ScanDownloadedFiles (bool mirror_mode) /* For local installs, if there is no src and no bin, the version * is unavailable */ - if (!i->accessible () && !pkgsrcver.accessible () - && *i != pkg.installed) + if (*i != pkg.installed + && !i->accessible () && !pkgsrcver.accessible () ) { +// Henri: innihilate the info that has been collected from setup.ini? DO NOT! + #if 0 if (pkg.prev == *i) pkg.prev = packageversion (); if (pkg.curr == *i) pkg.curr = packageversion (); if (pkg.exp == *i) pkg.exp = packageversion (); + #endif pkg.versions.erase (i++); /* For now, leave the source version alone */ } diff --git a/package_meta.h b/package_meta.h index b24d4fc..3d2b775 100644 --- a/package_meta.h +++ b/../src2/package_meta.h @@ -105,12 +105,14 @@ p
Re: Package choosing algorithm ...
On 2016-03-22 10:52, Corinna Vinschen wrote: On Mar 21 13:40, Houder wrote: [snip] Apparently specifying the -m option makes a difference ... It shouldn't but apparently it does, oh well. https://cygwin.com/acronyms#PGA anybody? Hi Corinna, More experimenting ... including building and testing setup.exe All the time using "Install from Local Directory" ... - copied cygwin-2.5.0-0.8.tar.xz to cygwin-2.5.0-0.9.tar.xz - modified setup.ini cygwin: substituted 0.9 for TEST - using my test version of setup.exe, I found out that the current version of the cygwin tarball (setup.ini) HAD to be in my repo ... or else setup would not offer the new test version for install ... - I modified package_meta.h accordingly - see below (my new test version is offered for install after the modification) - next I downloaded the current version of the cygwin tarball and moved the file to appropriate place in my repo. Using the official version of setup, I verified that my test version is now offered for install. Please, keep in mind: I am NOT announcing here that I found the bug ... Please, keep that in mind (inexperienced C++ developer and all that). I am sure more testing is required ... Regards, Henri == package_meta.h: std::string action_caption () const; packageversion trustp (bool _default, trusts const t) const { /* If the user chose "test" and a "test" version is available, return it. */ if (t == TRUST_TEST && exp) return exp; /* Are we looking for the default version and does the installed version have a higher version number than the "curr" package? This means the user has installed a "test" version, or built her own version newer than "curr". Rather than pulling the user back to "curr", we install "test" if a "test" version is available and the version number is higher, or we stick to "installed" if not. This reflects the behaviour of `yum update' on Fedora. */ // Henri //Log (LOG_PLAIN) << "HENRI:" << "\t" << this->name << "\t" << _default << "\t" << prev << curr << exp << "\t" << installed << endLog; Log (LOG_PLAIN) << "HENRI:" << "\t" << this->name << " " \ << curr << " " << installed << " cmp(c,i): " << packageversion::compareVersions (curr, installed) << " " \ << installed << " " << exp << " cmp(i,e): " << packageversion::compareVersions (installed, exp) << endLog; // Henri: Oops, current (setup.ini) is NOT present in repo in my case ... //if (_default && curr && installed if (_default && installed && packageversion::compareVersions (curr, installed) < 0) { if (exp && packageversion::compareVersions (installed, exp) < 0) { Log (LOG_PLAIN) << "HENRI!" << "\t" << this->name << endLog; return exp; } return installed; } /* Otherwise, if a "curr" version exists, return "curr". */ if (curr) return curr; /* Otherwise return the installed version. */ return installed; } = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Package choosing algorithm ...
On Mar 21 13:40, Houder wrote: > All the time using "Install from Local Directory" ... > > - copied vim-minimal-7.4.1558-1.tar.xz to vim-minimal-7.4.1600-1.tar.xz > - copied cygwin-2.5.0-0.8.tar.xz to cygwin-2.5.0-0.9.tar.xz > - modified setup.ini > vim-minimal: substituted 1600 for current > cygwin: substituted 0.9 for TEST < > > Invoked setup: > - vim-minimal is offered for install > - cygwin is NOT (0.8 is shown with a interrogation sign (?) for size ... > > ==> Again: > - modified setup.ini > cygwin: substituted 0.9 for CURRENT < > > Invoked setup: > - vim-minimal is offered for install > - cygwin is ** NOW ** offered for install ... > > ==> Again: > - modified setup.ini > cygwin: substituted 0.9 for TEST < (as in the 1st test) > > Invoked setup: BUT THIS TIME SPECIFYING the -m option > - vim-minimal is offered for install > - cygwin is ** NOW ** offered for install ... > > Apparently specifying the -m option makes a difference ... It shouldn't but apparently it does, oh well. https://cygwin.com/acronyms#PGA anybody? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: Package choosing algorithm ...
On 2016-03-21 11:00, Corinna Vinschen wrote: On Mar 20 17:54, Houder wrote: Hi Corinna, A year ago you changed the "package choosing algorithm" ... [snip] I'm using "Install from Local Directory: all the time (having a local mirror dir) and it works fine for me. No idea what has gone different in your scenario. If there's a bug, we're certainly open to get fixes :) Hi Corinna, Achim, Yaakov ... (did I miss anyone? :-) Did some more experimenting ... All the time using "Install from Local Directory" ... - copied vim-minimal-7.4.1558-1.tar.xz to vim-minimal-7.4.1600-1.tar.xz - copied cygwin-2.5.0-0.8.tar.xz to cygwin-2.5.0-0.9.tar.xz - modified setup.ini vim-minimal: substituted 1600 for current cygwin: substituted 0.9 for TEST < Invoked setup: - vim-minimal is offered for install - cygwin is NOT (0.8 is shown with a interrogation sign (?) for size ... ==> Again: - modified setup.ini cygwin: substituted 0.9 for CURRENT < Invoked setup: - vim-minimal is offered for install - cygwin is ** NOW ** offered for install ... ==> Again: - modified setup.ini cygwin: substituted 0.9 for TEST < (as in the 1st test) Invoked setup: BUT THIS TIME SPECIFYING the -m option - vim-minimal is offered for install - cygwin is ** NOW ** offered for install ... Apparently specifying the -m option makes a difference ... Regards, Henri -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Package choosing algorithm ...
On 2016-03-21 11:00, Corinna Vinschen wrote: On Mar 20 17:54, Houder wrote: Hi Corinna, A year ago you changed the "package choosing algorithm" ... [snip] I'm using "Install from Local Directory: all the time (having a local mirror dir) and it works fine for me. No idea what has gone different in your scenario. If there's a bug, we're certainly open to get fixes :) ... it works for you ... that is odd (yes, I am surprised, as I first did look for the cause of the problem at my side) From the above I understand that you specify -m option when calling setup ... Do you? Is that the only option that you use? (just asking, because I experimented with a different (higher) version number for the cygwin tar ball when, but it did not make a difference). Anyone using setup in the same way as I do and not experiencing my problem? Regards, Henri -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Package choosing algorithm ...
On Mar 20 17:54, Houder wrote: > Hi Corinna, > > A year ago you changed the "package choosing algorithm" ... > > https://sourceware.org/ml/cygwin/2015-02/msg00034.html > - [ANNOUNCEMENT] Updated: setup.exe (Release 2.864) -- by Corinna Vinschen > Here C. announces the modification of the "package choosing algorithm" > (and the introduction of the "mirror mode") > > When I started setup (2.873) and selected "Download without Installing", it > offered me to download version 2.5.0-0.8 of cygwin (installed: 2.5.0-0.7). > > Splendid! > > However when I restarted setup and selected "Install from Local Directory", > it did NOT offer me to install version 2.5.0-0.8 ... > > (Meaning, I had to restart setup AGAIN, and force it to install the last > version of cygwin - a bit annoying, as you wrote in [1]) > > Did your modification only apply to the "Download without Installing" part > of setup? (i.e. downloading from internet) I'm using "Install from Local Directory: all the time (having a local mirror dir) and it works fine for me. No idea what has gone different in your scenario. If there's a bug, we're certainly open to get fixes :) Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Package choosing algorithm ...
Hi Corinna, A year ago you changed the "package choosing algorithm" ... https://sourceware.org/ml/cygwin/2015-02/msg00034.html - [ANNOUNCEMENT] Updated: setup.exe (Release 2.864) -- by Corinna Vinschen Here C. announces the modification of the "package choosing algorithm" (and the introduction of the "mirror mode") When I started setup (2.873) and selected "Download without Installing", it offered me to download version 2.5.0-0.8 of cygwin (installed: 2.5.0-0.7). Splendid! However when I restarted setup and selected "Install from Local Directory", it did NOT offer me to install version 2.5.0-0.8 ... (Meaning, I had to restart setup AGAIN, and force it to install the last version of cygwin - a bit annoying, as you wrote in [1]) Did your modification only apply to the "Download without Installing" part of setup? (i.e. downloading from internet) (honestly, I do not remember whether or not setup acted consistently in both modes after your modification) Did I miss something? Regards, Henri [1] https://cygwin.com/ml/cygwin-apps/2015-01/msg00126.html - Setup patch to keep test version if test version installed - by Corinna Vinschen Start of discussion ... = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple