Re: Package choosing algorithm ... more clarification

[Houder]

For those interested, see attached file (chooser_overview.txt).

Regards,
Henrichooser_overview.txt:

2016-04-05 // Another attempt to create an overview ...

 = precondition
 - assume a "database" which reflects the info from setup.ini ... 
(packagedb::packages)
- it list each package and every version of that pkg ...
 - this database even tells us what version has been installed if that is the 
case (reflecting installed.db)
 - furthermore we (setup.exe) are able to ascertain whether the tarball, 
associated with a version, is available
   or not -- the source code of setup.exe uses the term "accessible" for 
availability

 = D w/o I (Download without Installing)
 - setup.exe should show us ALL the pkgs and every version of that pkg ...
- assume all tarballs are available (stable mirror)

 = I f LD (Install from Local Directory)
 - setup.exe should show us the installed pkgs and the versions of that pkg 
that are not installed, but are available
   in the repo ...
- the tarball of an installed version may or may not be available in the 
repo ...
 - furthermore it should show us each non-installed pkg if at least one version 
of that pkg is available in the repo

ChooserPage::OnInit has the following call flow in both cases (D w/o I and I f 
LD)

ChooserPage::OnInit
  packagemeta::ScanDownLoadedFiles
  packagedb::setExistence
  packagemeta::trustp(true, ...

 = D w/o I

 // (pkg.installed == version) => pkg is installed
 - ScanDownloadedFiles # method name is a misnomer here
! (pkg is installed || a version is acccessible) => erase version from 
packagemeta.versions
- here all versions will be kept (stable mirror) in packagemeta.versions
 - setExistence
! (pkg is installed || a version is acccessible) => erase pkg from packages
- the test whether the pkg is installed or not, is useless here ...
- ... as all pkgs (i.e. all versions of each pkg) are available 
(accessible): UNmodified database
 End result: setup.up will show us all the pkgs and every version of each pkg 
...

 = I f LD

 // (pkg.installed == version) => pkg is installed
 - ScanDownloadedFiles
! (pkg is installed || a version is accessible) => erase version from 
packagemeta.versions
- not-installed pkgs: available versions will be kept in 
packagemeta.versions
- installed pkgs: the installed version will be kept
- installed pkgs: non-available versions will be removed from 
packagemeta.versions
 - setExistence
! (pkg is installed || a version is accessible) => erase pkg from packages
- a not-installed pkg, of which at least one version is available: this pkg 
will be kept in the database
  (versions will show the available versions)
- installed pkgs: these pkgs will be kept in the database
  (versions will show the installed version and available versions that are 
not installed)
 End result: setup.exe will show us all the available versions of each pkg, if 
it has not been installed yet, and all
 the pkgs (both the installed version and available versions that are not 
installed) which have been installed ...

- Ignoring the issue of source tarballs ...

Each "package entry" (p, c and/or t) in setup.ini is represeneted by an object 
of type packagemeta ...

 - class variable packages (of type packagedb): std::map 
   or: a mapping from a packagename to a pointer to an object of type 
packagemeta

A simplified view of the data members of class packagemeta is as follows:

 - prev of type packageversion
 - curr ditto
 - exp ditto
 - installed ditto
 - versions of type std::set 
-
prev, curr, exp and installed reflect the "static" part of a "package entry": 
these data members reflect the info from both
setup.ini and installed.db ...
"Nullifying" these data members, as currently happens in ScanDownLoadedFiles is 
... stupid?

versions tells what versions of the pkg are available (accessible); versions is 
managed by ScanDownLoadedFiles ...

Attempting to grasp the general flow of computation:

packagemeta::ScanDownLoadedFiles =>
for each pkg // from packagedb::packages
  for each version // from metapackage::versions
[ .scan() ] // packageversion::scan() -- simplified!
// (pkg.installed == version) => pkg is installed
if ! (pkg is installed || version is accessible) // packagemeta::installed, 
packageversion::accessible()
erase version from versions
Note: a version is NOT erased if the version is installed OR if the version is 
available

packagemeta::accessible => // called by setExistence
for each version // from metapackage::versions
  if version is accessible // packageversion::accessible()
return true // pkg is available in (local) repo
  return false

packageversion::accessible =>
_packageversion::accessible =>
...

packagedb::setExistence
for each pkg // from packagedb::packages
  // (pkg.installed == version) => pkg is installed
  if ! (pkg is installed || a version is accessible) // packagemeta::installed, 

Re: Package choosing algorithm ...

[Houder]

Title: Re: Package choosing agorithm ...

Hi Corinna,

(Of course I am formally addressing the project leader, but Achim, Jon, 
Yaakov,

 others? are welcome to correct me if I am wrong below).

D w/o I = Download without Install
I f LD  = Install from Local Directory

As a reminder: I reported that a new test release of Cygwin is NOT 
offered

for install (having installed the previous test release) in my case.

https://cygwin.com/ml/cygwin/2016-03/msg00425.html

Relevant in my case is:

 - I am not using a local mirror ...
 - but a "partially populated Local Directory"
 - I do not invoke setup.exe with the -m option
 - my local directory does not have the current release of Cygwin

As you replied "it works for me" and "not using the -m option should not
make a difference", I decided to study the source code of setup.exe.

As I had never seen the source code of setup before, I started my study 
where

you had "left off" modifying (about a year ago).

Subsequently, I reported back to the list, announcing that I found the 
bug,

but was not sure I had.

   https://cygwin.com/ml/cygwin/2016-03/msg00480.html

I was wrong.

My patch did work, but for the wrong reasons.

So does your modification of packagemeta::trustp ...

Here is another attempt of mine (attempt? I seriously doubt whether 
there is

someone which fully understands the source code in all its details ...)

You will find 2 patch files attached. These patch files modify 
(correct):


 1. packagemeta::ScanDownLoadedFiles and
 2. packagemeta::trustp

(yes, I do realize, that this correction to setup, is NOT important 
enough to
 most of you - using a local mirror and all that; consequently, do 
whatever you

 like with my post - I am merely providing info to the list).

My argument for the correction is as follows:

 - packagedb::packages, a class variable, is the "package database" that 
is

   used by setup.
 - basically, it is a list of pairs (mapping between the packagename and 
its

   entry in setup.ini) (one where installed.db has been "integrated").
 - formally:
- typedef std::map  packagecollection;
- static packagecollection packages;
 - packagemeta is the type that describes a "package entry" from 
setup.ini
 - an object of type packagemeta does not only describe the previous, 
current
   and experimental (if applicable) lines from a package entry in 
setup.ini,
   it also describes the installed version of a package, as if 
installed.db is

   part of setup.ini.
--
 - bottom-line: the prev, curr, exp and installed data member of 
packagemeta
   are "static": these data members represent the info from setup.ini 
(and

   installed.db) and they should never be "nullified".
 - yet, that is exactly what happens in 
metapackage::ScanDownLoadedFiles.

--
 - packagemeta ALSO has a "not so static" data member, called versions; 
like

   packages, the contents of versions is "variable"
- versions, a set of type packageversion, contains the versions of 
the
  package that are available (the source code speaks about 
"accessible")
- available (in my simplistic view) means whether or not the tarball 
is

  present in the (local) repo
- unfortunately, though versions specifies which versions are 
available,

  it does not tell us whether the previous, current or experimental
  version of a package is available or not
--
 - bottom-line: version numbers can be compared if that info is present 
in
   setup.ini; however, although the info may be present, it does not 
tell us

   that the associated tarball is available (neither does versions).
 - consequently, metapackage:trustp does not only have to verify that
   setup.ini has the required info, it also has to verify that the 
tarball

   is available, before it can give the "go-ahead".

Lastly: No, I have not tested (been able) my modification for each 
possible
configuration out there (for instance, I do not install source 
tarballs).


Regards,
Henri

=diff --git a/package_meta.cc b/package_meta.cc
index 34ff78c..ff787a8 100644
--- a/package_meta.cc
+++ b/../src2/package_meta.cc
@@ -683,15 +683,18 @@ packagemeta::ScanDownloadedFiles (bool mirror_mode)
 	  /* For local installs, if there is no src and no bin, the version
 	   * is unavailable
 	   */
-	  if (!i->accessible () && !pkgsrcver.accessible ()
-	  && *i != pkg.installed)
+	  if (*i != pkg.installed
+	  && !i->accessible () && !pkgsrcver.accessible () )
 	{
+// Henri: innihilate the info that has been collected from setup.ini? DO NOT!
+	  #if 0
 	  if (pkg.prev == *i)
 		pkg.prev = packageversion ();
 	  if (pkg.curr == *i)
 		pkg.curr = packageversion ();
 	  if (pkg.exp == *i)
 		pkg.exp = packageversion ();
+	  #endif
 	  pkg.versions.erase (i++);
 	  /* For now, leave the source version alone */
 	}
diff --git a/package_meta.h b/package_meta.h
index b24d4fc..3d2b775 100644
--- a/package_meta.h
+++ 

Re: Package choosing algorithm ...

[Houder]

On 2016-03-22 10:52, Corinna Vinschen wrote:

On Mar 21 13:40, Houder wrote:


[snip]


Apparently specifying the -m option makes a difference ...


It shouldn't but apparently it does, oh well.

https://cygwin.com/acronyms#PGA anybody?


Hi Corinna,

More experimenting ... including building and testing setup.exe

All the time using "Install from Local Directory" ...

 - copied cygwin-2.5.0-0.8.tar.xz to cygwin-2.5.0-0.9.tar.xz
 - modified setup.ini
cygwin: substituted 0.9 for TEST

 - using my test version of setup.exe, I found out that the current 
version of
   the cygwin tarball (setup.ini) HAD to be in my repo ... or else setup 
would

   not offer the new test version for install ...
 - I modified package_meta.h accordingly - see below
   (my new test version is offered for install after the modification)
 - next I downloaded the current version of the cygwin tarball and moved
   the file to appropriate place in my repo.
   Using the official version of setup, I verified that my test version 
is now

   offered for install.

Please, keep in mind: I am NOT announcing here that I found the bug ... 
Please,

keep that in mind (inexperienced C++ developer and all that).

I am sure more testing is required ...

Regards,

Henri

==

package_meta.h:

  std::string action_caption () const;
  packageversion trustp (bool _default, trusts const t) const
  {
/* If the user chose "test" and a "test" version is available, 
return it. */

if (t == TRUST_TEST && exp)
  return exp;
/* Are we looking for the default version and does the installed 
version
   have a higher version number than the "curr" package?  This means 
the
   user has installed a "test" version, or built her own version 
newer
   than "curr".  Rather than pulling the user back to "curr", we 
install
   "test" if a "test" version is available and the version number is 
higher,
   or we stick to "installed" if not.  This reflects the behaviour 
of

   `yum update' on Fedora. */
// Henri
//Log (LOG_PLAIN) << "HENRI:" << "\t" << this->name << "\t" << _default 
<< "\t" << prev << curr << exp << "\t" << installed << endLog;

Log (LOG_PLAIN) << "HENRI:" << "\t" << this->name << " " \
<< curr << " " << installed << " cmp(c,i): " << 
packageversion::compareVersions (curr, installed) << " " \
<< installed << " " << exp << " cmp(i,e): " << 
packageversion::compareVersions (installed, exp) << endLog;


// Henri: Oops, current (setup.ini) is NOT present in repo in my case 
...

//if (_default && curr && installed
if (_default && installed
&& packageversion::compareVersions (curr, installed) < 0)
  {
if (exp && packageversion::compareVersions (installed, exp) < 0) 
{

Log (LOG_PLAIN) << "HENRI!" << "\t" << this->name << endLog;
  return exp;
}
return installed;
  }
/* Otherwise, if a "curr" version exists, return "curr". */
if (curr)
  return curr;
/* Otherwise return the installed version. */
return installed;
  }

=

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Package choosing algorithm ...

[Corinna Vinschen]

On Mar 21 13:40, Houder wrote:
> All the time using "Install from Local Directory" ...
> 
>  - copied vim-minimal-7.4.1558-1.tar.xz to vim-minimal-7.4.1600-1.tar.xz
>  - copied cygwin-2.5.0-0.8.tar.xz to cygwin-2.5.0-0.9.tar.xz
>  - modified setup.ini
> vim-minimal: substituted 1600 for current
> cygwin: substituted 0.9 for TEST <
> 
> Invoked setup:
>  - vim-minimal is offered for install
>  - cygwin is NOT (0.8 is shown with a interrogation sign (?) for size ...
> 
> ==> Again:
>  - modified setup.ini
> cygwin: substituted 0.9 for CURRENT <
> 
> Invoked setup:
>  - vim-minimal is offered for install
>  - cygwin is ** NOW ** offered for install ...
> 
> ==> Again:
>  - modified setup.ini
> cygwin: substituted 0.9 for TEST < (as in the 1st test)
> 
> Invoked setup: BUT THIS TIME SPECIFYING the -m option
>  - vim-minimal is offered for install
>  - cygwin is ** NOW ** offered for install ...
> 
> Apparently specifying the -m option makes a difference ...

It shouldn't but apparently it does, oh well.

https://cygwin.com/acronyms#PGA anybody?


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Package choosing algorithm ...

[Houder]

On 2016-03-21 11:00, Corinna Vinschen wrote:

On Mar 20 17:54, Houder wrote:

Hi Corinna,

A year ago you changed the "package choosing algorithm" ...


[snip]


I'm using "Install from Local Directory: all the time (having a local
mirror dir) and it works fine for me.  No idea what has gone different
in your scenario.  If there's a bug, we're certainly open to get fixes 
:)


Hi Corinna, Achim, Yaakov ... (did I miss anyone? :-)

Did some more experimenting ...

All the time using "Install from Local Directory" ...

 - copied vim-minimal-7.4.1558-1.tar.xz to vim-minimal-7.4.1600-1.tar.xz
 - copied cygwin-2.5.0-0.8.tar.xz to cygwin-2.5.0-0.9.tar.xz
 - modified setup.ini
vim-minimal: substituted 1600 for current
cygwin: substituted 0.9 for TEST <

Invoked setup:
 - vim-minimal is offered for install
 - cygwin is NOT (0.8 is shown with a interrogation sign (?) for size 
...


==> Again:
 - modified setup.ini
cygwin: substituted 0.9 for CURRENT <

Invoked setup:
 - vim-minimal is offered for install
 - cygwin is ** NOW ** offered for install ...

==> Again:
 - modified setup.ini
cygwin: substituted 0.9 for TEST < (as in the 1st test)

Invoked setup: BUT THIS TIME SPECIFYING the -m option
 - vim-minimal is offered for install
 - cygwin is ** NOW ** offered for install ...

Apparently specifying the -m option makes a difference ...

Regards,

Henri

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Package choosing algorithm ...

[Houder]

On 2016-03-21 11:00, Corinna Vinschen wrote:

On Mar 20 17:54, Houder wrote:

Hi Corinna,

A year ago you changed the "package choosing algorithm" ...


[snip]


I'm using "Install from Local Directory: all the time (having a local
mirror dir) and it works fine for me.  No idea what has gone different
in your scenario.  If there's a bug, we're certainly open to get fixes 
:)


... it works for you ... that is odd (yes, I am surprised, as I first 
did

look for the cause of the problem at my side)

From the above I understand that you specify -m option when calling 
setup

... Do you?

Is that the only option that you use? (just asking, because I 
experimented
with a different (higher) version number for the cygwin tar ball when, 
but

it did not make a difference).

Anyone using setup in the same way as I do and not experiencing my 
problem?


Regards,

Henri


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Package choosing algorithm ...

[Corinna Vinschen]

On Mar 20 17:54, Houder wrote:
> Hi Corinna,
> 
> A year ago you changed the "package choosing algorithm" ...
> 
> https://sourceware.org/ml/cygwin/2015-02/msg00034.html
>  - [ANNOUNCEMENT] Updated: setup.exe (Release 2.864) -- by Corinna Vinschen
> Here C. announces the modification of the "package choosing algorithm"
> (and the introduction of the "mirror mode")
> 
> When I started setup (2.873) and selected "Download without Installing", it
> offered me to download version 2.5.0-0.8 of cygwin (installed: 2.5.0-0.7).
> 
> Splendid!
> 
> However when I restarted setup and selected "Install from Local Directory",
> it did NOT offer me to install version 2.5.0-0.8 ...
> 
> (Meaning, I had to restart setup AGAIN, and force it to install the last
>  version of cygwin - a bit annoying, as you wrote in [1])
> 
> Did your modification only apply to the  "Download without Installing" part
> of setup? (i.e. downloading from internet)

I'm using "Install from Local Directory: all the time (having a local
mirror dir) and it works fine for me.  No idea what has gone different
in your scenario.  If there's a bug, we're certainly open to get fixes :)


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature