Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 09/01/2018 15:49, Ken Brown wrote: On 1/9/2018 10:37 AM, Ken Brown wrote: I just did a quick test, trying to uninstall B in the situation above, and it didn't work as expected. Even though "Uninstall A" was the first solution, A didn't get uninstalled. In case you want to replicate this, what I actually tried was uninstalling ImageMagick, which is required, directly or indirectly, by asymptote, dblatex, and xmlto. Solution 1 was to uninstall these three but it didn't happen. Doh. solver_take_solution() modifies the task list (and I went to the trouble of refactoring things to make it available to it), but I didn't then re-solve with the modified task list. Also, looking at this again, it looks like the solver places solutions which remove a task at the end of of the solution list, so this should be the default (This matches the previous behaviour, where the default is to accept dependencies i.e. if you try to remove a package required by other packages, the default solution should be to cancel the removal) Maybe we need more complex criteria to identify the default, but this seems to work in my limited testing. I've rebased and updated the topic/libsolv branch.
Re: [PATCH setup v3] Query the user if a corrupt local file is found
On 10/01/2018 02:59, Ken Brown wrote: Also reorganize package validation. Move the size-validation code in download.cc and the hash-validation code in install.cc into new member functions of the packagesource class. Add a bool member 'validated' to the class to make sure that the checking is done only once. Change download.cc:check_for_cached() so that it offers to delete a corrupt package file instead of throwing an exception. The latter previously caused a fatal error when check_for_cached() was called from do_download_thread and download_one. Now we get a fatal error only if the user chooses not to delete the file. Also make check_for_cached() check the hash of the file in addition to the size. Similarly, check the hash in addition to the size after downloading a file. This looks great, thanks. One minor comment, but please apply. diff --git a/res.rc b/res.rc index 82a9757..9648871 100644 --- a/res.rc +++ b/res.rc @@ -550,7 +550,8 @@ BEGIN IDS_DOWNLOAD_INCOMPLETE_EXIT "Download incomplete. Check %s for details" IDS_INSTALL_ERROR "Installation error (%s), Continue with other packages?" IDS_INSTALL_INCOMPLETE "Installation incomplete. Check %s for details" -IDS_CORRUPT_PACKAGE "Package file %s has a corrupt local copy, please remove and retry." +IDS_CORRUPT_PACKAGE "Package %s has a corrupt local copy, please remove and retry." +IDS_QUERY_CORRUPT "The file %s is corrupt. Delete it?." Perhaps this should say something like "Delete it and download again?" IDS_SKIP_PACKAGE "%s\nDo you want to skip this package ?" IDS_UNCAUGHT_EXCEPTION "Fatal Error: Uncaught Exception\nThread: %s\nType: %s\nMessage: %s" IDS_UNCAUGHT_EXCEPTION_WITH_ERRNO "Fatal Error: Uncaught Exception\nThread: %s\nType: %s\nMessage: %s\nAppErrNo: %d"
Re: Planned setup.ini changes for early 2018
On 10/01/2018 23:27, David Stacey wrote: On 10/01/18 22:44, Jon Turney wrote: * Add depends: to version descriptions * De-duplicate source archives * Migrate from setup.hint to pvr.hint in release area That all sounds good to me. Regarding the de-duplication of source files: will sources for 'noarch' packages remain in 'noarch', or will they move to 'src' as well? I have no plans to do this, although I can see it is a logical next step.
Re: Planned setup.ini changes for early 2018
On 12/01/2018 18:11, Peter A. Castro wrote: On Fri, 12 Jan 2018, Jon Turney wrote: On 11/01/2018 17:53, Peter A. Castro wrote: On Wed, 10 Jan 2018, Jon Turney wrote: * Add depends: to version descriptions This is a version-specific list of required packages (as opposed to requires:, which is per-package, and contains the union of the dependencies for all versions). I believe that historical setup versions will either ignore, or can handle depends: (just containing package names, without version relations) relatively sanely (see [1] et seq. for details). As long as the behaviour you outline in [1] is consistent, then that means I should be able to use older Setup with newer setup.ini, yes? That will be useful for people who use the Time Machine. :) Yes, this doesn't break backwards compatibility. Excellent! I'm not sure what reasons there are to use older setup on a circa from after the date of this change. It's not quite as strange as you might think. Older environments can't necessary run newer Setup (winXP anyone? :), but, on occasion, need some package from a later circa that's associated with a newer, but incompatable, Setup version. Yes, yes, I know, "don't do that". Caveat emptor and all that. Ok. You could also use setup flag --allow-unsupported-windows in that situation. I hope this means that the time machine can/will preserve depends: lines. It does. 'setup.ini' is preserved as originally distributed from cygwin.com (and, of course, it's mirrors). Oh. I had assumed for some reason that setup.ini was regenerated for each circa. Can I ask you to consider preserving the setup.ini.sig etc. as well? * De-duplicate source archives Source archives which are identical[2] between x86 and x86_64 will be moved to paths starting src/ in the release area. I'm not quite visualizing this, Do you mean there will be a new directory name 'src' that will be on the same level as 'x86' and 'x86_64' or will it be higher up the directory tree? It matters to me This means src/ on the same level as x86/ x86_64/ and noarch/. Thanks for the confirmation. I'll need to make some adjustments to the Time Machine for this, but should be trivial. Do you have a schedule of when you will start pushing this change out? No schedule, but not imminent, and certainly after the other items on this list. I'll let you know closer to the time. for the Time Machine as I'll need to be able to re-create the same directory hierarchy for each circa. It's a selectively automated process currently, but I need to know what symlink to place where. :) Doing post-hoc de-duplication is unfortunate, but worthwhile given the potential size saving in mirrors (see [3] et seq.), until cygport can be taught how to make suitable source packages (which has several unresolved issues, also discussed at [3], [4] et seq.). Will this be done en masse upon the next setup.ini build cycle, or over time as each new package is updated? Having a massive amount of "new" packages show up under a "new" directory named 'src' will be quite a lot for the mirrors to absorb all at once. For the Time Machine it might effectively be double as I have to maintain the old hierarchy as well as the new one (under the assumption that you'd apply the de-duplication retroactively). The process will be gradual and retroactive. Ok. Thanks for confirming. I'll watch for a mighty "gulp" in my pull logs. :) I'm not quite sure yet how it's going to apply to new uploads. Often x86 and x86_64 packages are uploaded separately, so either it happens asynchronously, after the last upload of the pair has been accepted, or I defer accepting and deduping the upload until both are uploaded. What of cases where the version for x86 and x86_64 uploaded aren't the same (it happens)? Will those be skipped or am I misunderstanding how this dedup process will work? Strictly, I guess this should be disallowed, since there aren't any good reasons for the source packages to be different. However, since there are suggestions that there are some packages where they can't be same due to shortcomings in cygport, I guess this case should be treated the same as currently.
Re: Planned setup.ini changes for early 2018
Jon Turney writes: > * De-duplicate source archives > > Source archives which are identical[2] between x86 and x86_64 will be > moved to paths starting src/ in the release area. You keep acting like using a new src/ hierarchy was already decided upon, yet I don't remember it even getting discussed. So, can we have that discussion before decision, please? To repeat, I think a separate src/ hierarchy would be a mistake, given where we are today, although it has a certain appeal from a greenfield perspective. But more to the point, there is nothing special about source archives (except that they're not architecture specific) that warrants putting them into their own hierarchy, IMHO. If they are going to get moved on deduplication (there are good reasons to move them and these likely outweight the benefits of just deleting them in one side of the hierarchy), we already have a place for that and that place is noarch/. That keeps everything in working order for folks who for whatever reason mirror the repository, even if the only mirror one architecture. If we also agree to actually move the files from a specific architecture branch to noarch/, any mirror operator can pre-populate the noarch/ hierarchy in order to minimize the amount of data to get synchronized (I suggest using x86_64). Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: Planned setup.ini changes for early 2018
Achim Gratz writes: > Put these in noarch/ and arrange for setup to be able to install both > from noarch/ and $arch/ so that we can also de-duplicate things like > debuginfo and documentation down the road. I forgot to add that this is already possible by injecting extra packages: say you deduplicate a debuginfo package into a noarch/ part (the source files) and the $arch/ part, then you just need to inject a dependency to the noarch package into the archful one and setup does the right thing. It would probably be a good idea to have a special category for such packages in order for them to not prompt the user for their installation. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 1/13/2018 9:14 AM, Jon Turney wrote: On 09/01/2018 15:49, Ken Brown wrote: On 1/9/2018 10:37 AM, Ken Brown wrote: I just did a quick test, trying to uninstall B in the situation above, and it didn't work as expected. Even though "Uninstall A" was the first solution, A didn't get uninstalled. In case you want to replicate this, what I actually tried was uninstalling ImageMagick, which is required, directly or indirectly, by asymptote, dblatex, and xmlto. Solution 1 was to uninstall these three but it didn't happen. Doh. solver_take_solution() modifies the task list (and I went to the trouble of refactoring things to make it available to it), but I didn't then re-solve with the modified task list. Also, looking at this again, it looks like the solver places solutions which remove a task at the end of of the solution list, so this should be the default (This matches the previous behaviour, where the default is to accept dependencies i.e. if you try to remove a package required by other packages, the default solution should be to cancel the removal) Maybe we need more complex criteria to identify the default, but this seems to work in my limited testing. I noticed two things: 1. I think the code you added to PrereqPage::OnNext() also needs to be added to PrereqPage::OnBack(). 2. We should probably remove, or at least reword, the dire warning about accepting the default solutions. I'm not sure we want to "strongly recommend" the default solution over the other solution(s). I guess what we really want to say is that we strongly recommend resolving the problems before continuing. Ken
Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 2018-01-13 12:56, Ken Brown wrote: > On 1/13/2018 9:14 AM, Jon Turney wrote: >> On 09/01/2018 15:49, Ken Brown wrote: >>> On 1/9/2018 10:37 AM, Ken Brown wrote: I just did a quick test, trying to uninstall B in the situation above, and it didn't work as expected. Even though "Uninstall A" was the first solution, A didn't get uninstalled. >>> >>> In case you want to replicate this, what I actually tried was uninstalling >>> ImageMagick, which is required, directly or indirectly, by asymptote, >>> dblatex, and xmlto. Solution 1 was to uninstall these three but it didn't >>> happen. >> >> Doh. >> >> solver_take_solution() modifies the task list (and I went to the trouble of >> refactoring things to make it available to it), but I didn't then re-solve >> with the modified task list. >> >> Also, looking at this again, it looks like the solver places solutions which >> remove a task at the end of of the solution list, so this should be the >> default >> >> (This matches the previous behaviour, where the default is to accept >> dependencies i.e. if you try to remove a package required by other packages, >> the default solution should be to cancel the removal) >> >> Maybe we need more complex criteria to identify the default, but this seems >> to >> work in my limited testing. > > I noticed two things: > > 1. I think the code you added to PrereqPage::OnNext() also needs to be added > to > PrereqPage::OnBack(). > > 2. We should probably remove, or at least reword, the dire warning about > accepting the default solutions. I'm not sure we want to "strongly recommend" > the default solution over the other solution(s). I guess what we really want > to > say is that we strongly recommend resolving the problems before continuing. For users who only run setup and use programs, a dire warning and strong recommendations are appropriate. Alternatives are to also remove all packages dependent on the package to be removed, or lastly, to remove only the requested package, leaving the installation inconsistent. Those alternatives would have to be presented to the user for selection, then executed. Anything else requiring the user to resolve would require a FAQ entry explaining what that meant, what diagnosis and actions would be required, and that would probably generate emails from users asking what they should do. Better to allow the solver to resolve issues and just let users choose straightforward alternatives, with the view of trying to keep the installation consistent, unless explicitly overridden, such as to test an alternative implementation of a dependency installed outside of setup. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 1/13/2018 4:29 PM, Brian Inglis wrote: On 2018-01-13 12:56, Ken Brown wrote: 2. We should probably remove, or at least reword, the dire warning about accepting the default solutions. I'm not sure we want to "strongly recommend" the default solution over the other solution(s). I guess what we really want to say is that we strongly recommend resolving the problems before continuing. For users who only run setup and use programs, a dire warning and strong recommendations are appropriate. Alternatives are to also remove all packages dependent on the package to be removed, or lastly, to remove only the requested package, leaving the installation inconsistent. Those alternatives would have to be presented to the user for selection, then executed. Anything else requiring the user to resolve would require a FAQ entry explaining what that meant, what diagnosis and actions would be required, and that would probably generate emails from users asking what they should do. Better to allow the solver to resolve issues and just let users choose straightforward alternatives, with the view of trying to keep the installation consistent, unless explicitly overridden, such as to test an alternative implementation of a dependency installed outside of setup. The current situation on the topic/libsolv branch is the following. Suppose A requires B and the user asks to uninstall B. They will get a problem report showing two possible solutions: 1. Uninstall A. 2. (default) Don't uninstall B. If they uncheck 'Accept default solutions' and select 'Next', they'll get a warning that says "We strongly recommend that you accept the default solutions. Some packages may not work properly if you don't. Are you sure you want to proceed?" This is misleading insofar as it implies that something bad will happen if the user prefers to solve the problem by uninstalling A. What is true is that some packages may not work properly if the user answers 'Yes'. I think we should be able to find wording that is accurate while still making it clear that we recommend going back and fixing the problem. I don't yet have a good candidate for that wording. Ken
Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 1/13/2018 5:55 PM, Ken Brown wrote:
On 1/13/2018 4:29 PM, Brian Inglis wrote:
On 2018-01-13 12:56, Ken Brown wrote:
2. We should probably remove, or at least reword, the dire warning about
accepting the default solutions. I'm not sure we want to "strongly
recommend"
the default solution over the other solution(s). I guess what we
really want to
say is that we strongly recommend resolving the problems before
continuing.
For users who only run setup and use programs, a dire warning and strong
recommendations are appropriate.
Alternatives are to also remove all packages dependent on the package
to be
removed, or lastly, to remove only the requested package, leaving the
installation inconsistent. Those alternatives would have to be
presented to the
user for selection, then executed.
Anything else requiring the user to resolve would require a FAQ entry
explaining
what that meant, what diagnosis and actions would be required, and
that would
probably generate emails from users asking what they should do.
Better to allow the solver to resolve issues and just let users choose
straightforward alternatives, with the view of trying to keep the
installation
consistent, unless explicitly overridden, such as to test an alternative
implementation of a dependency installed outside of setup.
The current situation on the topic/libsolv branch is the following.
Suppose A requires B and the user asks to uninstall B. They will get a
problem report showing two possible solutions:
1. Uninstall A.
2. (default) Don't uninstall B.
If they uncheck 'Accept default solutions' and select 'Next', they'll
get a warning that says "We strongly recommend that you accept the
default solutions. Some packages may not work properly if you don't.
Are you sure you want to proceed?"
This is misleading insofar as it implies that something bad will happen
if the user prefers to solve the problem by uninstalling A. What is
true is that some packages may not work properly if the user answers 'Yes'.
I think we should be able to find wording that is accurate while still
making it clear that we recommend going back and fixing the problem. I
don't yet have a good candidate for that wording.
Something like the attached might do the job.
Ken
>From d15d18dfa4db91416155385034bccf31be88ece3 Mon Sep 17 00:00:00 2001
From: Ken Brown
Date: Sat, 13 Jan 2018 18:50:14 -0500
Subject: [PATCH] Clarify the unsolved-problems warning
If the user unchecks the 'Accept default solutions' box and selects
'Next', don't imply that choosing a non-default solution would break
their system.
---
prereq.cc | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/prereq.cc b/prereq.cc
index a03e79b..4926c65 100644
--- a/prereq.cc
+++ b/prereq.cc
@@ -96,10 +96,9 @@ PrereqPage::OnNext ()
{
// breakage imminent! danger, danger
int res = MessageBox (h,
- "We strongly recommend that you accept the default solutions. "
- "Some packages may not work properly if you don't."
+ "Some packages may not work properly if you continue."
"\r\n\r\n"
- "Are you sure you want to proceed?",
+ "Are you sure you want to proceed (NOT RECOMMENDED)?",
"WARNING - Unsolved Problems",
MB_YESNO | MB_ICONEXCLAMATION | MB_DEFBUTTON2);
if (res == IDNO)
@@ -107,7 +106,7 @@ PrereqPage::OnNext ()
else
{
Log (LOG_PLAIN) <<
-"NOTE! User refused the default solutions! "
+"NOTE! User continued with unsolved problems! "
"Expect some packages to give errors or not function at all." <<
endLog;
// Change the solver's transaction list to reflect the user's
choices.
db.solution.db2trans();
--
2.15.1
Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 2018-01-13 17:00, Ken Brown wrote: > On 1/13/2018 5:55 PM, Ken Brown wrote: >> On 1/13/2018 4:29 PM, Brian Inglis wrote: >>> On 2018-01-13 12:56, Ken Brown wrote: 2. We should probably remove, or at least reword, the dire warning about accepting the default solutions. I'm not sure we want to "strongly recommend" the default solution over the other solution(s). I guess what we really want to say is that we strongly recommend resolving the problems before continuing. >>> >>> For users who only run setup and use programs, a dire warning and strong >>> recommendations are appropriate. >>> >>> Alternatives are to also remove all packages dependent on the package to be >>> removed, or lastly, to remove only the requested package, leaving the >>> installation inconsistent. Those alternatives would have to be presented to >>> the >>> user for selection, then executed. >>> >>> Anything else requiring the user to resolve would require a FAQ entry >>> explaining >>> what that meant, what diagnosis and actions would be required, and that >>> would >>> probably generate emails from users asking what they should do. >>> >>> Better to allow the solver to resolve issues and just let users choose >>> straightforward alternatives, with the view of trying to keep the >>> installation >>> consistent, unless explicitly overridden, such as to test an alternative >>> implementation of a dependency installed outside of setup. >> >> The current situation on the topic/libsolv branch is the following. Suppose A >> requires B and the user asks to uninstall B. They will get a problem report >> showing two possible solutions: >> >> 1. Uninstall A. >> 2. (default) Don't uninstall B. >> >> If they uncheck 'Accept default solutions' and select 'Next', they'll get a >> warning that says "We strongly recommend that you accept the default >> solutions. Some packages may not work properly if you don't. Are you sure >> you >> want to proceed?" >> >> This is misleading insofar as it implies that something bad will happen if >> the >> user prefers to solve the problem by uninstalling A. What is true is that >> some packages may not work properly if the user answers 'Yes'. >> >> I think we should be able to find wording that is accurate while still making >> it clear that we recommend going back and fixing the problem. I don't yet >> have a good candidate for that wording. > > Something like the attached might do the job. Just saying "Unsolved problems" does not tell the user what they did and what impact it will have, and invites a FAQ entry for Setup - Unsolved problems. Could we please be more explicit in the UI and the logs about the action being taken and the impact: instead of "Unsolved problems" and "Some packages" maybe something more like "uninstalling package U will break packages P1...", and instead of "default solutions" maybe something more like "recommended actions"? That shows and records the action and impact for diagnosis and remediation when users do something unintended and need to undo it, changing the messaging from the implementation model to the user's model, and giving users and maintainers enough information to diagnose what will or did happen, and how to undo that. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
Re: [PATCH setup 00/14] Use libsolv, solve all our problems... (WIP)
On 1/13/2018 8:52 PM, Brian Inglis wrote: On 2018-01-13 17:00, Ken Brown wrote: On 1/13/2018 5:55 PM, Ken Brown wrote: On 1/13/2018 4:29 PM, Brian Inglis wrote: On 2018-01-13 12:56, Ken Brown wrote: 2. We should probably remove, or at least reword, the dire warning about accepting the default solutions. I'm not sure we want to "strongly recommend" the default solution over the other solution(s). I guess what we really want to say is that we strongly recommend resolving the problems before continuing. For users who only run setup and use programs, a dire warning and strong recommendations are appropriate. Alternatives are to also remove all packages dependent on the package to be removed, or lastly, to remove only the requested package, leaving the installation inconsistent. Those alternatives would have to be presented to the user for selection, then executed. Anything else requiring the user to resolve would require a FAQ entry explaining what that meant, what diagnosis and actions would be required, and that would probably generate emails from users asking what they should do. Better to allow the solver to resolve issues and just let users choose straightforward alternatives, with the view of trying to keep the installation consistent, unless explicitly overridden, such as to test an alternative implementation of a dependency installed outside of setup. The current situation on the topic/libsolv branch is the following. Suppose A requires B and the user asks to uninstall B. They will get a problem report showing two possible solutions: 1. Uninstall A. 2. (default) Don't uninstall B. If they uncheck 'Accept default solutions' and select 'Next', they'll get a warning that says "We strongly recommend that you accept the default solutions. Some packages may not work properly if you don't. Are you sure you want to proceed?" This is misleading insofar as it implies that something bad will happen if the user prefers to solve the problem by uninstalling A. What is true is that some packages may not work properly if the user answers 'Yes'. I think we should be able to find wording that is accurate while still making it clear that we recommend going back and fixing the problem. I don't yet have a good candidate for that wording. Something like the attached might do the job. Just saying "Unsolved problems" does not tell the user what they did and what impact it will have, and invites a FAQ entry for Setup - Unsolved problems. "WARNING - Unsolved Problems" is simply the caption on the message box. Could we please be more explicit in the UI and the logs about the action being taken and the impact: instead of "Unsolved problems" and "Some packages" maybe something more like "uninstalling package U will break packages P1...", and We were already explicit when we displayed the prerequisites page with a problem report. There's no need to repeat it. What we're discussing right now is what warning we should give if the user, *after seeing the problem report*, unchecks the 'Accept default solutions' box and selects 'Next'. instead of "default solutions" maybe something more like "recommended actions"? I've already explained why I think it would be misleading to say "recommended" rather than "default". In the scenario above with packages A and B, the user may be trying to pare down their installation by uninstalling some packages. They try to uninstall B, we warn them that this will break A, and we offer them two possible solutions. It's perfectly reasonable for us to present "Keep B" as the default solution, especially since, as Jon pointed out, that's what we currently do. But I don't think it's reasonable for us to *recommend* keeping B without knowing what the user wanted to accomplish. Maybe uninstalling A better fits their needs. That shows and records the action and impact for diagnosis and remediation when users do something unintended and need to undo it, changing the messaging from the implementation model to the user's model, and giving users and maintainers enough information to diagnose what will or did happen, and how to undo that. The current code in the topic/libsolv branch already does this. Maybe it could do it better. It would be great if you would test it and suggest improvements. Ken
