Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-14 Thread Luca Berra

On Thu, Oct 14, 2010 at 09:44:39AM +0100, Buchan Milne wrote:

On Saturday, 9 October 2010 10:10:36 Renaud MICHEL wrote:

On samedi 09 octobre 2010 at 05:45, andré wrote :
> Note that configuration files that have been changed from the
> installation default are often already saved.  (Generally ".old" is
> appended to the configuration file name, sometimes ".new" to the new
> configuration file.)

But here you are only talking about system-wide configuration files, which
are known of rpm as they are part of the package and marked as config
files. But what about user specific configuration files?
For the easy kind, where a program will have a single configuration file
(or dedicated directory), a pre-inst script could find it in the home of
each users and backup them. But you have cases of programs which have
configuration scattered in multiple shared directories (like KDE), or even
non-deterministic configuration files and it can become very tricky to find
all the files to backup.


A snapshot-based solution (e.g. on btrfs) would mean you don't have to "find 
all the files".


uhm. we are speaking about user files, what if reverting a snapshot
makes users loose a whole day of work.

L.

--
Luca Berra -- bl...@vodka.it


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-14 Thread Sinner from the Prairy
Frank Griffin wrote:

>> Now that's getting very hackish.
>> I'd rather not have many programs wrapped in scripts that would do some
>> magic on my home dir under the hood. Because with such a "solution" the
>> programs would be always wrapped, even if you never do a rollback.
>> How would such script detect that it actually was a rollback and it
>> should do his magic on the config files.
>>
> 
> The wrapper script would be specific to the package version which
> provided it.  If it finds a saved config file with a name matching its
> own version, it restores it and deletes the saved one.  If it doesn't,
> it does nothing.

I am still not liking anything messing with users' home


Salut,
Sinner



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-13 Thread Frank Griffin
Renaud MICHEL wrote:
> On Wednesday 13 octobre 2010 at 23:34, Frank Griffin wrote :
>   
>> One way of doing it might be, having identified packages that require
>> this sort of support, to wrap the executables with scripts that do this
>> the next time a user runs the software before the real executable is
>> launched.
>> 
> Now that's getting very hackish.
> I'd rather not have many programs wrapped in scripts that would do some 
> magic on my home dir under the hood. Because with such a "solution" the 
> programs would be always wrapped, even if you never do a rollback.
> How would such script detect that it actually was a rollback and it should 
> do his magic on the config files.
>   

The wrapper script would be specific to the package version which
provided it.  If it finds a saved config file with a name matching its
own version, it restores it and deletes the saved one.  If it doesn't,
it does nothing.

> What would happen if the user did not run that program between the update 
> and the rollback?
>   

Nothing, because the new wrapper script would never have been executed
to save a previous version, so the restored old wrapper script would not
find anything to restore.



> It seems the complexity is not worth the benefit, and those scripts are 
> likely to not be well tested and might make things worse if things are not 
> like they expected.
>   

That's a pretty broad statement, especially considering the complexity
of some of the wrapper scripts we already have.  This is not that
complex, and the benefit of finding a solution is considerable, based on
the previous posts/relies in this thread.



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-13 Thread Renaud MICHEL
On Wednesday 13 octobre 2010 at 23:34, Frank Griffin wrote :
> One way of doing it might be, having identified packages that require
> this sort of support, to wrap the executables with scripts that do this
> the next time a user runs the software before the real executable is
> launched.

Now that's getting very hackish.
I'd rather not have many programs wrapped in scripts that would do some 
magic on my home dir under the hood. Because with such a "solution" the 
programs would be always wrapped, even if you never do a rollback.
How would such script detect that it actually was a rollback and it should 
do his magic on the config files.
What would happen if the user did not run that program between the update 
and the rollback?

It seems the complexity is not worth the benefit, and those scripts are 
likely to not be well tested and might make things worse if things are not 
like they expected.

-- 
Renaud Michel


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-13 Thread Frank Griffin
Michael Scherer wrote:
>
> touching /home is forbidden in %pre and %post per policy.
>
> 1) you cannot be sure that every user are available ( ie, using
> ldap/nis )
>
> 2) you cannot be sure that someone is not already using the
> directory/config file ( ie, shared home by nfs, or multiuser system )
>
> 3) you cannot be sure that the home is mounted ( auto mounter based on
> login, etc )
>   

Excellent point; I hadn't thought of that. 

One way of doing it might be, having identified packages that require
this sort of support, to wrap the executables with scripts that do this
the next time a user runs the software before the real executable is
launched.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-13 Thread Michael Scherer
Le samedi 09 octobre 2010 à 11:10 +0200, Renaud MICHEL a écrit :
> On samedi 09 octobre 2010 at 05:45, andré wrote :
> > Note that configuration files that have been changed from the 
> > installation default are often already saved.  (Generally ".old" is 
> > appended to the configuration file name, sometimes ".new" to the new 
> > configuration file.)
> 
> But here you are only talking about system-wide configuration files, which 
> are known of rpm as they are part of the package and marked as config files.
> But what about user specific configuration files?
> For the easy kind, where a program will have a single configuration file (or 
> dedicated directory), a pre-inst script could find it in the home of each 
> users and backup them.

touching /home is forbidden in %pre and %post per policy.

1) you cannot be sure that every user are available ( ie, using
ldap/nis )

2) you cannot be sure that someone is not already using the
directory/config file ( ie, shared home by nfs, or multiuser system )

3) you cannot be sure that the home is mounted ( auto mounter based on
login, etc )


-- 
Michael Scherer



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-10 Thread Frank Griffin
andré wrote:
> But wouldn't most users of cooker just take some packages from cooker
> to install on their stable release ?  Even if it's a computer used
> only for testing purposes ?

That's an ancient issue in MDV.  Installing cooker packages on stable
releases doesn't work, because compilers and versions change
incompatibly in ways that rpm can't or doesn't track, e.g. kernel versions.





Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-10 Thread Frank Griffin
andré wrote:
>
> Note that insisting that all backports depend on the same versions of
> required packages as the distro release for which the backport is made
> will considerably reduce one potential source of problems.  

This isn't really necessary.  rpm/urpmi can handle multiple versions of
packages and the dependencies.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Frank Griffin
Marc Paré wrote:
>
> I guess both of your suggestions would work André and Margot.
>
> So, this sounds like it could be something that could be worked out. I
> would prefer, as a user, Margot's process. Of course the user would
> have to be able to see the usage and re-claimed space due to his use
> or Backports. Just to keep informed.
All of this is way more complicated than it needs to be. 

The original proposal included a provision for leaving the post-release
backports packages on the backports mirror when a newer backport of that
package arrived, so that the older version could be downloaded if needed
for a rollback. 

I left out the option to cache old packages on the user's system
precisely to avoid arguments about space, but if you combine the two,
i.e. check the user's system and go to the mirror if it's not there,
then you can do something relatively simple, such as automatically
deleting local versions of packages after a certain amount of time. 

No need to prompt the user since the old package can always be retrieved
from the mirror, and it will be transparent to the user (except for
slightly slower response) where the package came from.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Frank Griffin
Luca Berra wrote:
> On Sat, Oct 09, 2010 at 11:10:36AM +0200, Renaud MICHEL wrote:
>>
>> But here you are only talking about system-wide configuration files,
>> which are known of rpm as they are part of the package and marked as
>> config files.
>> But what about user specific configuration files?
>
> +1
> this is the very problem why i believe rollbacks are not that easy
>
The proposal included a recommendation for this, having the package
scripts do the equivalent of an "rpmnew"/"rpmold", but with
package-specific names.  Note that in a rollback, any changes made to
the newer file would be lost, but that's unavoidable in any case since
they may not be supported by the old format.

Fortunately, the number of packages that do this reformatting is
relatively small, and some of them (like NetBeans) create the newer
versions in new-release-specific subdirectories, leaving the old ones
intact.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Luca Berra

On Sat, Oct 09, 2010 at 06:48:40PM +, Dick Gevers wrote:

On Sat, 9 Oct 2010 18:42:25 +0200, Luca Berra wrote about Re: [Mageia-dev]
Proposal: Updating released versions (long post):


--repackage has been dropped from urpmi since it has been dropped from
rpm 4.6


Will you post it in Mdv Bugzilla? 


what for?


To request / remember that urpmi should be fixed in this respect ?


https://qa.mandriva.com/show_bug.cgi?id=61226

--
Luca Berra -- bl...@vodka.it


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Ahmad Samir
On 9 October 2010 20:48, Dick Gevers  wrote:
> On Sat, 9 Oct 2010 18:42:25 +0200, Luca Berra wrote about Re: [Mageia-dev]
> Proposal: Updating released versions (long post):
>
>>>>--repackage has been dropped from urpmi since it has been dropped from
>>>>rpm 4.6
>>>
>>>Will you post it in Mdv Bugzilla?
>>
>>what for?
>
> To request / remember that urpmi should be fixed in this respect ?
>
> What else?    :)
>
> Cheers,
> =Dick Gevers=
>

There is nothing to fix, rpm doesn't have the --repackage option any
more; urpmi can't use a feature that doesn't exist

-- 
Ahmad Samir


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Dick Gevers
On Sat, 9 Oct 2010 18:42:25 +0200, Luca Berra wrote about Re: [Mageia-dev]
Proposal: Updating released versions (long post):

>>>--repackage has been dropped from urpmi since it has been dropped from
>>>rpm 4.6
>>
>>Will you post it in Mdv Bugzilla? 
>
>what for?

To request / remember that urpmi should be fixed in this respect ?

What else?:)

Cheers,
=Dick Gevers=


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Luca Berra

On Sat, Oct 09, 2010 at 01:16:09PM +, Dick Gevers wrote:

On Sat, 9 Oct 2010 12:28:44 +0200, Luca Berra wrote about Re: [Mageia-dev]
Proposal: Updating released versions (long post):


--repackage has been dropped from urpmi since it has been dropped from
rpm 4.6


Will you post it in Mdv Bugzilla? 


what for?

--
Luca Berra -- bl...@vodka.it


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Thomas Backlund

andré skrev 9.10.2010 17:41:

BTW, a feature to blacklist the installation of a particular version of
a package would be very useful.  I.e. an option "never install this
package" or "never select automatically this package".
This would prevent a package found to be problematic on a particular
system from being accidentally later installed on the system in question.


urpmi already have /etc/urpmi/skip.list,
so I guess a gui part is only missing.

--
Thomas



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread andré

Frank Griffin a écrit :

Part of the recent thread about what the desirable release cycle should
be devolved into a discussion of how backports works and whether or not
it's suitable.  I'd like to examine this issue.

The current urpmi repository architecture serves purposes that were
meaningful to Mandriva.  It segregates main from contrib for statement
of support reasons, it separates non-free from main for philosophical
reasons, and it separates restricted from main for legal and business
reasons.

This works pretty well for cooker, where you either want a particular
category of package considered or you don't, but the reuse of this model
for updates, backports, and testing in released versions isn't as good a
fit.

The root of the problem is that the user base is different.  Users of
cooker want the latest and greatest of everything, and have accepted
that if something breaks in cooker, it may stay broken for awhile.
Users of released versions vary all over the map, from people who never
want anything to change to people who want some specific updates to
people who want everything new but want it stable.  Users of cooker
rarely think about security updates, because in grabbing everything
available constantly, the security updates are "just there".  With users
of released versions, they may have to opt-in for security updates, and
usually want to treat other updates differently.
   
But wouldn't most users of cooker just take some packages from cooker to 
install on their stable release ?  Even if it's a computer used only for 
testing purposes ?

I'd like to propose the following model for updating released versions:

1) Users should not have to see, except in minor ways, the different
repositories.  Urpmi may see them, and advanced or ideologically polar
users may care about them (e.g. free vs non-free), but most users
won't.  Instead, let urpmi or rpmdrake have knowledge about all
repositories whether enabled or not, and display the offerings with an
icon, tooltip, or extra column that indicates the status of the package.
   

Lets assume that the GUI rpmdrake is used.
Instead of hiding the repositories (or partially hiding them as now done),
let's display the current selection on entering rpmdrake, BEFORE taking 
the time to load and analyse the list of packages installed and 
available on the selected repositories.

Add a checkbox to update for each repository.
They should be listed with a brief description, or have a description 
available with context help.

The user then adjusts the selection as desired, before the loading/analysis.
If the user doesn't want to change the selection, they just press return.
Quick, easy - and the user always knows the options available and what 
is selected.
And indeed, in the package list displayed it would be useful to have a 
column for the source repository.

2) The update tool we give these users should distinguish between
security updates and backports/testing, but present them both.  This is
very much like the Windows Update model, where all available fixes are
divided into "Critical System Updates" and "Software Updates".  We don't
really have the same support constraints as Mandriva, and there's no
need to automatically disable backports across the board, and not even
present the backports as possibilities.
   
With a column displaying the source repository, only an option to 
display updates is necessary, instead of the current "all updates", 
"security updates", "correction of anomolies", and "general updates".

I would keep backports separate, as they are necessarily more problematic.
However I would make the remaining display options with checkboxes : 
all, meta packages, graphic applications, updates, and backports.

as well, I would add all/installed/non-installed options to each line.
All these options to remember the last selection, for ease of use.
This way, the user sees all the display options in an easy to follow table.

3) Users should be able to enable options for each category
independently.  Most users would probably want security updates applied
automatically, but would want to be notified of availability of
backports or testing and choose those manually.
   
For automatic selection of packages to install, that is how it works 
now, which is fine.  But the user should always have the option to 
override the automatic selection.


For example, gedit, the gnome default editor has an extention where the 
latest version doesn't work properly for my purposes.  An automatic 
update installs the newer version, every time there is any type of 
update to gedit.  I had to use certain options of the command-line tool 
to override this to reinstall the working version.  And have to ensure 
that the non-working extension is not installed during updates.


BTW, a feature to blacklist the installation of a particular version of 
a package would be very useful.  I.e. an option "never install this 
package" or "never select automatically this package".
Th

Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread andré

Luca Berra a écrit :


On Sat, Oct 09, 2010 at 11:10:36AM +0200, Renaud MICHEL wrote:

On samedi 09 octobre 2010 at 05:45, andré wrote :
Note that configuration files that have been changed from the 
installation default are often already saved.  (Generally ".old" is 
appended to the configuration file name, sometimes ".new" to the new 
configuration file.)


But here you are only talking about system-wide configuration files, 
which are known of rpm as they are part of the package and marked as 
config files.

But what about user specific configuration files?
For the easy kind, where a program will have a single configuration 
file (or dedicated directory), a pre-inst script could find it in the 
home of each users and backup them. But you have cases of programs 
which have configuration scattered in multiple shared directories 
(like KDE), or even non-deterministic configuration files and it can 
become very tricky to find all the files to backup.
And you have the really hard kind, where the same configuration file 
may be shared by different packages. For example, plasma applet 
configurations are stored in a few ~/.kde4/share/config/plasma* files.
Good point.  So user-specific configuration files would have to be taken 
into account.

BTW, there is a similar complication for some Gnome packages.
So it would take a lot more time and effort to accommodate such rollbacks.
However most packages would be much simpler to reliably roll back.
So it seems useful that packages be marked as suitable for rollback (or 
not).

+1
this is the very problem why i believe rollbacks are not that easy

Because the rollback will be automatic and must be very reliable, it is 
not something to be taken lightly.  A malfunctioning rollback would be 
considerably worse than the current manual rollback process.


Note that insisting that all backports depend on the same versions of 
required packages as the distro release for which the backport is made 
will considerably reduce one potential source of problems.  As well, if 
a new version of a particular application has important format changes, 
perhaps there should be a policy of not making backports for it.
So candidates for rollbacks would be mostly restricted to applications 
developped and tested for the current release of the distro.

Thus the utility of automated rollbacks would be considerably reduced.

Another 2 cents :)

- André (andre999)


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Dick Gevers
On Sat, 9 Oct 2010 12:28:44 +0200, Luca Berra wrote about Re: [Mageia-dev]
Proposal: Updating released versions (long post):

>--repackage has been dropped from urpmi since it has been dropped from
>rpm 4.6

Will you post it in Mdv Bugzilla? 

TIA
=Dick Gevers=


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Luca Berra

On Sat, Oct 09, 2010 at 11:10:36AM +0200, Renaud MICHEL wrote:

On samedi 09 octobre 2010 at 05:45, andré wrote :
Note that configuration files that have been changed from the 
installation default are often already saved.  (Generally ".old" is 
appended to the configuration file name, sometimes ".new" to the new 
configuration file.)


But here you are only talking about system-wide configuration files, which 
are known of rpm as they are part of the package and marked as config files.

But what about user specific configuration files?
For the easy kind, where a program will have a single configuration file (or 
dedicated directory), a pre-inst script could find it in the home of each 
users and backup them. But you have cases of programs which have 
configuration scattered in multiple shared directories (like KDE), or even 
non-deterministic configuration files and it can become very tricky to find 
all the files to backup.
And you have the really hard kind, where the same configuration file may be 
shared by different packages. For example, plasma applet configurations are 
stored in a few ~/.kde4/share/config/plasma* files.


+1
this is the very problem why i believe rollbacks are not that easy

--
Luca Berra -- bl...@vodka.it


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Luca Berra

On Sat, Oct 09, 2010 at 08:05:08AM +0100, Margot wrote:

There are already various options which can be run with urpmi, such
as --noclean and --repackage (see man urpmi) and perhaps these
could be incorporated into the GUI - with full simple explanations
for the user on exactly what will happen if those options are
chosen.

man page is wrong,
--repackage has been dropped from urpmi since it has been dropped from
rpm 4.6

--
Luca Berra -- bl...@vodka.it


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Renaud MICHEL
On samedi 09 octobre 2010 at 05:45, andré wrote :
> Note that configuration files that have been changed from the 
> installation default are often already saved.  (Generally ".old" is 
> appended to the configuration file name, sometimes ".new" to the new 
> configuration file.)

But here you are only talking about system-wide configuration files, which 
are known of rpm as they are part of the package and marked as config files.
But what about user specific configuration files?
For the easy kind, where a program will have a single configuration file (or 
dedicated directory), a pre-inst script could find it in the home of each 
users and backup them. But you have cases of programs which have 
configuration scattered in multiple shared directories (like KDE), or even 
non-deterministic configuration files and it can become very tricky to find 
all the files to backup.
And you have the really hard kind, where the same configuration file may be 
shared by different packages. For example, plasma applet configurations are 
stored in a few ~/.kde4/share/config/plasma* files.

-- 
Renaud Michel


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Marc Paré

Le 2010-10-09 03:05, Margot a écrit :

On Sat, 09 Oct 2010 02:15:18 -0400
andré  wrote:


Marc Paré a écrit :


Le 2010-10-08 23:45, andré a écrit :

Frank Griffin a écrit :

Marc Paré wrote:

Thanks. So this thread is to see if there were a possibility
to programme a more efficient roll-back option so that it
would be more "aware" of the previous "dependencies" needs
for the previous version. Having "double dependencies" is
not so much of a problem, it is the rollback to a previous
version where the dependency confusion may occur, and, ONLY,
if an upgraded type of "dependency" thread had been
installed. (Sorry I may have used the wrong terms in the
last sentence).

Well, sort of. It's not an issue of efficiency, but of
convenience and just making it possible to do without
resorting to manual use of the rpm
command.

The rpm command "knows" that a new version replacing the old
version supplies the same things that the old one did, so it
will quietly allow the upgrade. It will also do what we need,
i.e. go the other way and replace a newer version with an
older one if you use the --oldpackage keyword. We just need
urpmi to support its use


One thing that could facilitate this process, if the computer
has a lot of free disk space, is to rename the files being
removed (copying the configuration files), instead of erasing
them. Although they will probably have to be erased
eventually, since no computer has unlimited disk space.
Keeping them long enough that a roll-back is no longer
probable could be workable. Then a roll-back could be done
very quickly, since the files are already on disk, and
presumably reliably. Of course, if new data has been entered,
and the format has been changed, this could be problematic.
Note that configuration files that have been changed from the
installation default are often already saved. (Generally
".old" is appended to the configuration file name, sometimes
".new" to the new configuration file.) This of course adds the
maintenance task of removing the old files at some point - it
could be done automatically according to some criteria, or the
user could have to do it manually, perhaps after being
prompted about it.

(This rollback capability occurs with Microsoft products. The
saved files are only removed manually, on user initiative,
which partly explains the bloated size of a Microsoft
installation over time.)

If renaming-instead-of-deleting is implemented, perhaps a "do
not keep old program files (useful if limited disk space)"
checkbox option would be useful for computers with less free
disk space. Of course how much disk space is usable to save
old programs on a computer depends on the disk space usage for
other purposes over time.

my 2 cents :)

- André (andre999)


Not sure about this process. Instead of making it easier for a
user, this would now make it more difficult to do and add
another layer of knowledge for the new user. It would have to
be a little more seamless than this.

If there were a way at setup to establish the amount of
remaining disk space at installation time, and if there were
enough space to allow rollbacks without compromising the
installation, then I guess the rollback could then be
activated. The user could then be advised at this point that
this was activated. If there was not enought disk space, a
message could warn the user that software rollbacks would not
be possible for lack lack of diskspace.

The problem is not establishing the current free disk space, but
how much to leave for use as temporary disk space for other
applications. For example, if an enduser likes editing numerous
large video files at the same time (maybe he makes movies), he
could need a very large amount of temporary free disk space.
Another user, with the same programs installed, might do
primarily word processing and Internet, and only occasionally
edit small videos, thus only needing a relatively small amount of
temporary disk space. Of course, there could be an automatic
default, adjustable via a configuration file.


I guess then, in the MCC, if a user used the Backports and
installed backported software, the rollback amount of diskspace
could also be monitored at this level with perhaps an option to
delete old programs that are now working well in their updated
form.

This sort of makes sense -- but it is not only the newly
installed program which is of concern, but also other programs
which may have the same dependancies (not counting the versions).
It could take a considerable time before these other programs are
executed, so it becomes a bit tricky.
Probably why Microsoft decided to keep such programs by default.

Essentially that is why I would prefer backports to use versions
of dependancies which correspond to the distro release.  A bit
more work for packagers, but a much more stable system.
Then the rollback system would only affect the backported program
and any programs directly dependant on that version.  The problem
becomes much simpler.
Once the backpor

Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-09 Thread Margot
On Sat, 09 Oct 2010 02:15:18 -0400
andré  wrote:

> Marc Paré a écrit :
> >
> > Le 2010-10-08 23:45, andré a écrit :
> >> Frank Griffin a écrit :
> >>> Marc Paré wrote:
>  Thanks. So this thread is to see if there were a possibility
>  to programme a more efficient roll-back option so that it
>  would be more "aware" of the previous "dependencies" needs
>  for the previous version. Having "double dependencies" is
>  not so much of a problem, it is the rollback to a previous
>  version where the dependency confusion may occur, and, ONLY,
>  if an upgraded type of "dependency" thread had been
>  installed. (Sorry I may have used the wrong terms in the
>  last sentence).
> >>> Well, sort of. It's not an issue of efficiency, but of
> >>> convenience and just making it possible to do without
> >>> resorting to manual use of the rpm
> >>> command.
> >>>
> >>> The rpm command "knows" that a new version replacing the old
> >>> version supplies the same things that the old one did, so it
> >>> will quietly allow the upgrade. It will also do what we need,
> >>> i.e. go the other way and replace a newer version with an
> >>> older one if you use the --oldpackage keyword. We just need
> >>> urpmi to support its use
> >>
> >> One thing that could facilitate this process, if the computer
> >> has a lot of free disk space, is to rename the files being
> >> removed (copying the configuration files), instead of erasing
> >> them. Although they will probably have to be erased
> >> eventually, since no computer has unlimited disk space.
> >> Keeping them long enough that a roll-back is no longer
> >> probable could be workable. Then a roll-back could be done
> >> very quickly, since the files are already on disk, and
> >> presumably reliably. Of course, if new data has been entered,
> >> and the format has been changed, this could be problematic.
> >> Note that configuration files that have been changed from the
> >> installation default are often already saved. (Generally
> >> ".old" is appended to the configuration file name, sometimes
> >> ".new" to the new configuration file.) This of course adds the
> >> maintenance task of removing the old files at some point - it
> >> could be done automatically according to some criteria, or the
> >> user could have to do it manually, perhaps after being
> >> prompted about it.
> >>
> >> (This rollback capability occurs with Microsoft products. The
> >> saved files are only removed manually, on user initiative,
> >> which partly explains the bloated size of a Microsoft
> >> installation over time.)
> >>
> >> If renaming-instead-of-deleting is implemented, perhaps a "do
> >> not keep old program files (useful if limited disk space)"
> >> checkbox option would be useful for computers with less free
> >> disk space. Of course how much disk space is usable to save
> >> old programs on a computer depends on the disk space usage for
> >> other purposes over time.
> >>
> >> my 2 cents :)
> >>
> >> - André (andre999)
> >
> > Not sure about this process. Instead of making it easier for a
> > user, this would now make it more difficult to do and add
> > another layer of knowledge for the new user. It would have to
> > be a little more seamless than this.
> >
> > If there were a way at setup to establish the amount of
> > remaining disk space at installation time, and if there were
> > enough space to allow rollbacks without compromising the
> > installation, then I guess the rollback could then be
> > activated. The user could then be advised at this point that
> > this was activated. If there was not enought disk space, a
> > message could warn the user that software rollbacks would not
> > be possible for lack lack of diskspace.
> The problem is not establishing the current free disk space, but
> how much to leave for use as temporary disk space for other
> applications. For example, if an enduser likes editing numerous
> large video files at the same time (maybe he makes movies), he
> could need a very large amount of temporary free disk space.
> Another user, with the same programs installed, might do
> primarily word processing and Internet, and only occasionally
> edit small videos, thus only needing a relatively small amount of
> temporary disk space. Of course, there could be an automatic
> default, adjustable via a configuration file.
> >
> > I guess then, in the MCC, if a user used the Backports and
> > installed backported software, the rollback amount of diskspace
> > could also be monitored at this level with perhaps an option to
> > delete old programs that are now working well in their updated
> > form.
> This sort of makes sense -- but it is not only the newly
> installed program which is of concern, but also other programs
> which may have the same dependancies (not counting the versions).
> It could take a considerable time before these other programs are 
> executed, so it becomes a bit tricky.
> Probably why Microsoft decided to keep such

Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread andré

Marc Paré a écrit :


Le 2010-10-08 23:45, andré a écrit :

Frank Griffin a écrit :

Marc Paré wrote:

Thanks. So this thread is to see if there were a possibility to
programme a more efficient roll-back option so that it would be more
"aware" of the previous "dependencies" needs for the previous version.
Having "double dependencies" is not so much of a problem, it is the
rollback to a previous version where the dependency confusion may
occur, and, ONLY, if an upgraded type of "dependency" thread had been
installed. (Sorry I may have used the wrong terms in the last 
sentence).

Well, sort of. It's not an issue of efficiency, but of convenience and
just making it possible to do without resorting to manual use of the 
rpm

command.

The rpm command "knows" that a new version replacing the old version
supplies the same things that the old one did, so it will quietly allow
the upgrade. It will also do what we need, i.e. go the other way and
replace a newer version with an older one if you use the --oldpackage
keyword. We just need urpmi to support its use


One thing that could facilitate this process, if the computer has a lot
of free disk space, is to rename the files being removed (copying the
configuration files), instead of erasing them. Although they will
probably have to be erased eventually, since no computer has unlimited
disk space. Keeping them long enough that a roll-back is no longer
probable could be workable.
Then a roll-back could be done very quickly, since the files are already
on disk, and presumably reliably. Of course, if new data has been
entered, and the format has been changed, this could be problematic.
Note that configuration files that have been changed from the
installation default are often already saved. (Generally ".old" is
appended to the configuration file name, sometimes ".new" to the new
configuration file.)
This of course adds the maintenance task of removing the old files at
some point - it could be done automatically according to some criteria,
or the user could have to do it manually, perhaps after being prompted
about it.

(This rollback capability occurs with Microsoft products. The saved
files are only removed manually, on user initiative, which partly
explains the bloated size of a Microsoft installation over time.)

If renaming-instead-of-deleting is implemented, perhaps a "do not keep
old program files (useful if limited disk space)" checkbox option would
be useful for computers with less free disk space.
Of course how much disk space is usable to save old programs on a
computer depends on the disk space usage for other purposes over time.

my 2 cents :)

- André (andre999)


Not sure about this process. Instead of making it easier for a user, 
this would now make it more difficult to do and add another layer of 
knowledge for the new user. It would have to be a little more seamless 
than this.


If there were a way at setup to establish the amount of remaining disk 
space at installation time, and if there were enough space to allow 
rollbacks without compromising the installation, then I guess the 
rollback could then be activated. The user could then be advised at 
this point that this was activated. If there was not enought disk 
space, a message could warn the user that software rollbacks would not 
be possible for lack lack of diskspace.
The problem is not establishing the current free disk space, but how 
much to leave for use as temporary disk space for other applications.
For example, if an enduser likes editing numerous large video files at 
the same time (maybe he makes movies), he could need a very large amount 
of temporary free disk space.  Another user, with the same programs 
installed, might do primarily word processing and Internet, and only 
occasionally edit small videos, thus only needing a relatively small 
amount of temporary disk space.
Of course, there could be an automatic default, adjustable via a 
configuration file.


I guess then, in the MCC, if a user used the Backports and installed 
backported software, the rollback amount of diskspace could also be 
monitored at this level with perhaps an option to delete old programs 
that are now working well in their updated form.
This sort of makes sense -- but it is not only the newly installed 
program which is of concern, but also other programs which may have the 
same dependancies (not counting the versions).
It could take a considerable time before these other programs are 
executed, so it becomes a bit tricky.

Probably why Microsoft decided to keep such programs by default.

Essentially that is why I would prefer backports to use versions of 
dependancies which correspond to the distro release.  A bit more work 
for packagers, but a much more stable system.
Then the rollback system would only affect the backported program and 
any programs directly dependant on that version.  The problem becomes 
much simpler.
Once the backported and dependant programs (which would be known in the 
dat

Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Marc Paré

Le 2010-10-08 23:45, andré a écrit :

Frank Griffin a écrit :

Marc Paré wrote:

Thanks. So this thread is to see if there were a possibility to
programme a more efficient roll-back option so that it would be more
"aware" of the previous "dependencies" needs for the previous version.
Having "double dependencies" is not so much of a problem, it is the
rollback to a previous version where the dependency confusion may
occur, and, ONLY, if an upgraded type of "dependency" thread had been
installed. (Sorry I may have used the wrong terms in the last sentence).

Well, sort of. It's not an issue of efficiency, but of convenience and
just making it possible to do without resorting to manual use of the rpm
command.

The rpm command "knows" that a new version replacing the old version
supplies the same things that the old one did, so it will quietly allow
the upgrade. It will also do what we need, i.e. go the other way and
replace a newer version with an older one if you use the --oldpackage
keyword. We just need urpmi to support its use


One thing that could facilitate this process, if the computer has a lot
of free disk space, is to rename the files being removed (copying the
configuration files), instead of erasing them. Although they will
probably have to be erased eventually, since no computer has unlimited
disk space. Keeping them long enough that a roll-back is no longer
probable could be workable.
Then a roll-back could be done very quickly, since the files are already
on disk, and presumably reliably. Of course, if new data has been
entered, and the format has been changed, this could be problematic.
Note that configuration files that have been changed from the
installation default are often already saved. (Generally ".old" is
appended to the configuration file name, sometimes ".new" to the new
configuration file.)
This of course adds the maintenance task of removing the old files at
some point - it could be done automatically according to some criteria,
or the user could have to do it manually, perhaps after being prompted
about it.

(This rollback capability occurs with Microsoft products. The saved
files are only removed manually, on user initiative, which partly
explains the bloated size of a Microsoft installation over time.)

If renaming-instead-of-deleting is implemented, perhaps a "do not keep
old program files (useful if limited disk space)" checkbox option would
be useful for computers with less free disk space.
Of course how much disk space is usable to save old programs on a
computer depends on the disk space usage for other purposes over time.

my 2 cents :)

- André (andre999)




Not sure about this process. Instead of making it easier for a user, 
this would now make it more difficult to do and add another layer of 
knowledge for the new user. It would have to be a little more seamless 
than this.


If there were a way at setup to establish the amount of remaining disk 
space at installation time, and if there were enough space to allow 
rollbacks without compromising the installation, then I guess the 
rollback could then be activated. The user could then be advised at this 
point that this was activated. If there was not enought disk space, a 
message could warn the user that software rollbacks would not be 
possible for lack lack of diskspace.


I guess then, in the MCC, if a user used the Backports and installed 
backported software, the rollback amount of diskspace could also be 
monitored at this level with perhaps an option to delete old programs 
that are now working well in their updated form.


I guess this would take a bit of coding. But at least the use of 
Backports would make a little more sense with a rollback option in case 
an updated software installation did not work out.


Marc



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread andré

Frank Griffin a écrit :

Marc Paré wrote:
   

Thanks. So this thread is to see if there were a possibility to
programme a more efficient roll-back option so that it would be more
"aware" of the previous "dependencies" needs for the previous version.
Having "double dependencies" is not so much of a problem, it is the
rollback to a previous version where the dependency confusion may
occur, and, ONLY, if an upgraded type of "dependency" thread had been
installed. (Sorry I may have used the wrong terms in the last sentence).
 

Well, sort of.  It's not an issue of efficiency, but of convenience and
just making it possible to do without resorting to manual use of the rpm
command.

The rpm command "knows" that a new version replacing the old version
supplies the same things that the old one did, so it will quietly allow
the upgrade.  It will also do what we need, i.e. go the other way and
replace a newer version with an older one if you use the --oldpackage
keyword.  We just need urpmi to support its use


One thing that could facilitate this process, if the computer has a lot 
of free disk space, is to rename the files being removed (copying the 
configuration files), instead of erasing them.  Although they will 
probably have to be erased eventually, since no computer has unlimited 
disk space.  Keeping them long enough that a roll-back is no longer 
probable could be workable.
Then a roll-back could be done very quickly, since the files are already 
on disk, and presumably reliably.  Of course, if new data has been 
entered, and the format has been changed, this could be problematic.
Note that configuration files that have been changed from the 
installation default are often already saved.  (Generally ".old" is 
appended to the configuration file name, sometimes ".new" to the new 
configuration file.)
This of course adds the maintenance task of removing the old files at 
some point - it could be done automatically according to some criteria, 
or the user could have to do it manually, perhaps after being prompted 
about it.


(This rollback capability occurs with Microsoft products.  The saved 
files are only removed manually, on user initiative, which partly 
explains the bloated size of a Microsoft installation over time.)


If renaming-instead-of-deleting is implemented, perhaps a "do not keep 
old program files (useful if limited disk space)" checkbox option would 
be useful for computers with less free disk space.
Of course how much disk space is usable to save old programs on a 
computer depends on the disk space usage for other purposes over time.


my 2 cents :)

- André (andre999)



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Frank Griffin
Marc Paré wrote:
>
> Thanks. So this thread is to see if there were a possibility to
> programme a more efficient roll-back option so that it would be more
> "aware" of the previous "dependencies" needs for the previous version.
> Having "double dependencies" is not so much of a problem, it is the
> rollback to a previous version where the dependency confusion may
> occur, and, ONLY, if an upgraded type of "dependency" thread had been
> installed. (Sorry I may have used the wrong terms in the last sentence).

Well, sort of.  It's not an issue of efficiency, but of convenience and
just making it possible to do without resorting to manual use of the rpm
command. 

The rpm command "knows" that a new version replacing the old version
supplies the same things that the old one did, so it will quietly allow
the upgrade.  It will also do what we need, i.e. go the other way and
replace a newer version with an older one if you use the --oldpackage
keyword.  We just need urpmi to support its use.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Marc Paré

Le 2010-10-08 16:32, Frank Griffin a écrit :

Marc Paré wrote:


So, in terms of space used for this, if you had to install all 6,
would this tax the system so much and risk filling up the hardrive
needlessly.


Not really, since the old versions would be removed when the new ones
were installed.  The behavior I described is not part of the proposal;
that's what happens today.



It not, if a rollback were done, could all 6 as well as the new F be
removed and the old version restored?


Yes, that's exactly what happens today.The problem is that today,
removing them may cause tons of other packages to have to be removed
because they require things that A-F provide.  This wasn't a problem on
upgrade, because the removal of the old and the addition of the new was
a single urpmi "transaction" (I put this in quotes because urpmi uses
"transaction" to mean something other than what I mean here), and urpmi
"knew" that the new versions supplied all the things that vanished when
the old versions were removed.  Today, rollbacks have to be done
manually - remove new, then install old.  Urpmi doesn't know at the time
of the removal that you're going to turn around and install the old
versions next.  It only sees that all the things that both the old and
new versions supply are about to disappear from your system, so it tells
you that you have to remove any other package which requires those things.


If this is possible, would this have an impact on devs preparing
Backport versions with rollbacks?


RPM dependencies aren't a problem.  Urpmi/urpme know all about them.
The only packaging changes would be for situations like that where a new
version of an application has changed a format of one of its files in
your home directory and the new version automatically converts the old
version of the file to the new format.  In that case, the package would
need install scriptlets that copied the old version somewhere so that it
could be restored at uninstall time, otherwise the old version of the
software won't be able to use the new file format.

The biggest chunk of development involved in the proposal is to make
urpmi do a rollback as a single operation, just as it does an upgrade.
This already exists, in a way; there is a facility called urpmi.recover
that does this type of thing.  Bit it's not really considered
mainstream, and I don't think it's been supported for a while.




Thanks. So this thread is to see if there were a possibility to 
programme a more efficient roll-back option so that it would be more 
"aware" of the previous "dependencies" needs for the previous version. 
Having "double dependencies" is not so much of a problem, it is the 
rollback to a previous version where the dependency confusion may occur, 
and, ONLY, if an upgraded type of "dependency" thread had been 
installed. (Sorry I may have used the wrong terms in the last sentence).


Marc



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Frank Griffin
Renaud MICHEL wrote:
> If you still have the old package (which could have been reconstructed with 
> the --repackage option of urpmi), rpm can upgrade to an older version using 
> the --oldpackage option. So, if the dependencies are sill met with the old 
> package you won't have the uninstall problem.
>
> I don't think urpmi support that, but if rpm can do it, it should not be to 
> hard to add it to urpmi.
>   
Exactly.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Frank Griffin
Marc Paré wrote:
>
> So, in terms of space used for this, if you had to install all 6,
> would this tax the system so much and risk filling up the hardrive
> needlessly.

Not really, since the old versions would be removed when the new ones
were installed.  The behavior I described is not part of the proposal;
that's what happens today.

>
> It not, if a rollback were done, could all 6 as well as the new F be
> removed and the old version restored?

Yes, that's exactly what happens today.The problem is that today,
removing them may cause tons of other packages to have to be removed
because they require things that A-F provide.  This wasn't a problem on
upgrade, because the removal of the old and the addition of the new was
a single urpmi "transaction" (I put this in quotes because urpmi uses
"transaction" to mean something other than what I mean here), and urpmi
"knew" that the new versions supplied all the things that vanished when
the old versions were removed.  Today, rollbacks have to be done
manually - remove new, then install old.  Urpmi doesn't know at the time
of the removal that you're going to turn around and install the old
versions next.  It only sees that all the things that both the old and
new versions supply are about to disappear from your system, so it tells
you that you have to remove any other package which requires those things.
>
> If this is possible, would this have an impact on devs preparing
> Backport versions with rollbacks?

RPM dependencies aren't a problem.  Urpmi/urpme know all about them. 
The only packaging changes would be for situations like that where a new
version of an application has changed a format of one of its files in
your home directory and the new version automatically converts the old
version of the file to the new format.  In that case, the package would
need install scriptlets that copied the old version somewhere so that it
could be restored at uninstall time, otherwise the old version of the
software won't be able to use the new file format.

The biggest chunk of development involved in the proposal is to make
urpmi do a rollback as a single operation, just as it does an upgrade. 
This already exists, in a way; there is a facility called urpmi.recover
that does this type of thing.  Bit it's not really considered
mainstream, and I don't think it's been supported for a while.



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Renaud MICHEL
On vendredi 08 octobre 2010 at 20:01, Frank Griffin wrote :
> The problem is that removing the new package, if it provides something
> that lots of things require, will want to uninstall a large number of
> packages uselessly, because it doesn't know that you're about to turn
> around and install something else that provides it.  All I'm suggesting
> is that
>(a) we make the older RPMs available somewhere
>(b) we automate the process to save three manual steps, and
>(c) we wrap them in an atomic transaction (or what looks like one to
> the user) to solve the problem above

If you still have the old package (which could have been reconstructed with 
the --repackage option of urpmi), rpm can upgrade to an older version using 
the --oldpackage option. So, if the dependencies are sill met with the old 
package you won't have the uninstall problem.

I don't think urpmi support that, but if rpm can do it, it should not be to 
hard to add it to urpmi.

-- 
Renaud Michel


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Marc Paré




Wherever this can be done, it usually is.  Multiple versions of
libraries can coexist in Linux, so if the new version requires new
libraries, there's no conflict.

The dependencies that cause problems are, for example,  the situation
where you can only have one of a certain service provider and its users
are bound to it by an API which changes between the release version and
the new version you want to install.  For example, if A, B, C, D, E all
use F, and the new version of C has been changed to use the
(incompatible) API of the new version of F, you'd have to install all 6.





So, in terms of space used for this, if you had to install all 6, would 
this tax the system so much and risk filling up the hardrive needlessly.


If not, would this cause your installation to misbehave.

It not, if a rollback were done, could all 6 as well as the new F be 
removed and the old version restored?


If this is possible, would this have an impact on devs preparing 
Backport versions with rollbacks?


Marc



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Frank Griffin
Marc Paré wrote:
> So, as to your proposal, I am in agreement with it. Maybe a suggestion
> (more of a question), seeing as the issues have to deal more with the
> dependencies of a "roll-back", what if, "Backports" would install,
> somehow the software updates in a self compartment kind of way (sorry
> my language terms may not be descriptive enough).

Michael hinted at that, either taking a copy-on-write (brtfs) copy of
the user's system for later restore, or having users install the update
on a VM copy of their system, but the image thing is good for only one
update at a time, and I don't think most users are going to re-install
in a VM to test a new package.  There really isn't a good way to install
in a "sandbox".

>
> Users know that repositories are activated and some are not at the
> time of install. I think that most users have a faint idea of this.
> But if Backports were to be used more at the user level, what is
> clear, is that, there should be a roll-back feature. If the roll-back
> feature, as a normal course of rolling-back will or could remove other
> softs due to dependencies, then maybe possible option would be to have
> the "Backport" software installations install all dependencies in such
> a way as to isolate them from the larger distro packages. This would
> in effect could make a secondary related-dependency file on the
> system. Then if a roll-back were done a Backported software package
> only the associated dependencies would be deleted.
>
> Could this be done?

Wherever this can be done, it usually is.  Multiple versions of
libraries can coexist in Linux, so if the new version requires new
libraries, there's no conflict.

The dependencies that cause problems are, for example,  the situation
where you can only have one of a certain service provider and its users
are bound to it by an API which changes between the release version and
the new version you want to install.  For example, if A, B, C, D, E all
use F, and the new version of C has been changed to use the
(incompatible) API of the new version of F, you'd have to install all 6.




Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Marc Paré

Le 2010-10-08 13:49, Frank Griffin a écrit :

Marc Paré wrote:



As it seems we keep going in circles on this Romain has arranged the
following so that the threads on this topic become more focussed:

I'd read the thread previously, and checked the wiki page, but this is a
different issue.  That wiki page is pretty much just a poll (who are
you, what do you do with your systems, what kind of release schedule do
you want).  This proposal is independent of whether there's a rolling
release or not; it's about presenting the update process to the user in
a different way.



Yup, I reread your proposal. You are right. It would be nice for you to 
add it to the wiki page. This is after all, as you said, one of the 
issues raised in the thread.




As to your proposal, the issue with "Backports" and lack of 
understanding form the users point of view is quite a large one. There 
should be a VERY descriptive info section explaining the purpose of 
"Backports". The lack of a descriptive expanation just did not make the 
"Backports" section of the MCC popular by users. We just assumed that 
when a newer version of our favourite soft. had come out, that all we 
had to do is wait for it to come up in the "you know, that red ball 
thingy in the tasbar on the right-bottom".


I am now making use of Backports and loving it! But also realize the 
dangers of using it.




So, as to your proposal, I am in agreement with it. Maybe a suggestion 
(more of a question), seeing as the issues have to deal more with the 
dependencies of a "roll-back", what if, "Backports" would install, 
somehow the software updates in a self compartment kind of way (sorry my 
language terms may not be descriptive enough).


Users know that repositories are activated and some are not at the time 
of install. I think that most users have a faint idea of this. But if 
Backports were to be used more at the user level, what is clear, is 
that, there should be a roll-back feature. If the roll-back feature, as 
a normal course of rolling-back will or could remove other softs due to 
dependencies, then maybe possible option would be to have the "Backport" 
software installations install all dependencies in such a way as to 
isolate them from the larger distro packages. This would in effect could 
make a secondary related-dependency file on the system. Then if a 
roll-back were done a Backported software package only the associated 
dependencies would be deleted.


Could this be done?

Sorry, I am not sure if this is written clearly enough.

Marc



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Frank Griffin
Luca Berra wrote:
> On Thu, Oct 07, 2010 at 07:49:29PM -0400, Frank Griffin wrote:
>> I'd like to propose the following model for updating released versions:
>>
>> 1) Users should not have to see, except in minor ways, the different
>> repositories.  Urpmi may see them, and advanced or ideologically polar
>> users may care about them (e.g. free vs non-free), but most users
> I object to this specific point, if what we are doing is free software
> that should be made clear to users. and i believe the user shold be
> asked wether they want to use only free software or not.

I have no objection to allowing users to check a box saying they want
free or mixed systems.  I just don't want them to have to mess with a
repository to do it.

> I just read it 3 times, and i still believe doing the above might
> prove to be a
> nightmare.
> rolling back a single, well identified change is a doable task.
> rolling back proceduraly a complex change becomes exponentially complex
> even for experienced system engineers, let alone a piece of software.

I agree that the work becomes exponentially complex, but not the concept
of what has to be done.  After all, what can be installed in an
automated way can be uninstalled. 

In any case, I would think that the primary use of this facility would
be to uninstall a single package.  Urpme already knows how to do that,
and (provided you have the older RPM around) urpmi knows how to install
the old one.  We expect users who want to roll back a package to do that
manually today.

The problem is that removing the new package, if it provides something
that lots of things require, will want to uninstall a large number of
packages uselessly, because it doesn't know that you're about to turn
around and install something else that provides it.  All I'm suggesting
is that
   (a) we make the older RPMs available somewhere
   (b) we automate the process to save three manual steps, and
   (c) we wrap them in an atomic transaction (or what looks like one to
the user) to solve the problem above


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Frank Griffin
Marc Paré wrote:
>
>
> As it seems we keep going in circles on this Romain has arranged the
> following so that the threads on this topic become more focussed:
I'd read the thread previously, and checked the wiki page, but this is a
different issue.  That wiki page is pretty much just a poll (who are
you, what do you do with your systems, what kind of release schedule do
you want).  This proposal is independent of whether there's a rolling
release or not; it's about presenting the update process to the user in
a different way.


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Michael Scherer
Le vendredi 08 octobre 2010 à 09:36 +0200, Luca Berra a écrit :
> On Thu, Oct 07, 2010 at 07:49:29PM -0400, Frank Griffin wrote:
> >
> >(Here's the biggie :-) )
> >4) We need to enhance the urpmi.recover functionality and bring it fully
> >into mainstream urpmi so that ANY PACKAGE CAN BE ROLLED BACK TO ITS
> >PREVIOUS VERSION (sorry for the caps).  If we don't want to be stuck
> I just read it 3 times, and i still believe doing the above might prove to be 
> a
> nightmare.
> rolling back a single, well identified change is a doable task.
> rolling back proceduraly a complex change becomes exponentially complex
> even for experienced system engineers, let alone a piece of software.

We could, using brtfs/lvm snapshots. But this may produce some hidden
side effects. Maybe we could do the reverse :
ie, we do a clone / snapshot for testing the upgrade ( maybe in a vm,
except a vm would not help to test real hardware issue ), and if people
are not satisfied , they do not put it in "production".

I think there was a presentation about rpm rollback  at FOSDEM this year
( or last year, maybe ), but I didn't really listened, as I do not
remember how it ended.

-- 
Michael Scherer



Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-08 Thread Luca Berra

On Thu, Oct 07, 2010 at 07:49:29PM -0400, Frank Griffin wrote:

I'd like to propose the following model for updating released versions:

1) Users should not have to see, except in minor ways, the different
repositories.  Urpmi may see them, and advanced or ideologically polar
users may care about them (e.g. free vs non-free), but most users

I object to this specific point, if what we are doing is free software
that should be made clear to users. and i believe the user shold be
asked wether they want to use only free software or not.


2) The update tool we give these users should distinguish between
security updates and backports/testing, but present them both.  This is

updates are not necessarily limited to security, but ok
backports is one thing, testing is a completely different one
the only use for testing is pulling out a specific (set of) packages for
QA purpose, no user should ever see those.
but the idea of separating the two in the user interface is sound.


(Here's the biggie :-) )
4) We need to enhance the urpmi.recover functionality and bring it fully
into mainstream urpmi so that ANY PACKAGE CAN BE ROLLED BACK TO ITS
PREVIOUS VERSION (sorry for the caps).  If we don't want to be stuck

I just read it 3 times, and i still believe doing the above might prove to be a
nightmare.
rolling back a single, well identified change is a doable task.
rolling back proceduraly a complex change becomes exponentially complex
even for experienced system engineers, let alone a piece of software.

i'll spend more time on doing qa on packages pushed to released distros.

L.


--
Luca Berra -- bl...@vodka.it


Re: [Mageia-dev] Proposal: Updating released versions (long post)

2010-10-07 Thread Marc Paré

Le 2010-10-07 19:49, Frank Griffin a écrit :

Part of the recent thread about what the desirable release cycle should
be devolved into a discussion of how backports works and whether or not
it's suitable.  I'd like to examine this issue.

The current urpmi repository architecture serves purposes that were
meaningful to Mandriva.  It segregates main from contrib for statement
of support reasons, it separates non-free from main for philosophical
reasons, and it separates restricted from main for legal and business
reasons.

This works pretty well for cooker, where you either want a particular
category of package considered or you don't, but the reuse of this model
for updates, backports, and testing in released versions isn't as good a
fit.

The root of the problem is that the user base is different.  Users of
cooker want the latest and greatest of everything, and have accepted
that if something breaks in cooker, it may stay broken for awhile.
Users of released versions vary all over the map, from people who never
want anything to change to people who want some specific updates to
people who want everything new but want it stable.  Users of cooker
rarely think about security updates, because in grabbing everything
available constantly, the security updates are "just there".  With users
of released versions, they may have to opt-in for security updates, and
usually want to treat other updates differently.

I'd like to propose the following model for updating released versions:

1) Users should not have to see, except in minor ways, the different
repositories.  Urpmi may see them, and advanced or ideologically polar
users may care about them (e.g. free vs non-free), but most users
won't.  Instead, let urpmi or rpmdrake have knowledge about all
repositories whether enabled or not, and display the offerings with an
icon, tooltip, or extra column that indicates the status of the package.

2) The update tool we give these users should distinguish between
security updates and backports/testing, but present them both.  This is
very much like the Windows Update model, where all available fixes are
divided into "Critical System Updates" and "Software Updates".  We don't
really have the same support constraints as Mandriva, and there's no
need to automatically disable backports across the board, and not even
present the backports as possibilities.

3) Users should be able to enable options for each category
independently.  Most users would probably want security updates applied
automatically, but would want to be notified of availability of
backports or testing and choose those manually.

(Here's the biggie :-) )
4) We need to enhance the urpmi.recover functionality and bring it fully
into mainstream urpmi so that ANY PACKAGE CAN BE ROLLED BACK TO ITS
PREVIOUS VERSION (sorry for the caps).  If we don't want to be stuck
with trying to reconcile our desire to QA some packages better than
others with some users' desires to at least *try* the newest stuff, then
we need to allow them to move forwards and backwards in the package
history as easily as possible.

Yes, I know this is problematic.  It means that we have to do a really
good job of getting dependencies right.  But if the dependencies *are*
right, then this should be doable.

It means that we need to expand the logic in urpmi that can currently
identify the packages that need to be uninstalled if some other package
is uninstalled so that it can take into account the package it will be
installing in its place (and the other older versions of packages that
it will require), and compare the two lists to produce a "diff".

It needs to decide which changes can be "quiet", e.g. "A" 1.3 requires
"B" 1.3" and "A" 1.2 requires "B" 1.2, so a request to replace "A" 1.3
with "A" 1.2 would cause a replacement of "B" 1.3 with "B" 1.2 in the
same transaction.  This may have a cascading effect.  In any event, the
user should be told what's going to be backlevelled, but specifically
*not* see the current urpmi list of everything that will have to be
removed if "A" 1.3 is removed if most of that stuff is simply going to
be replaced with its own previous versions.  In other words, rather than
tell the user that removing "A" 1.3 is going to remove half of KDE and
scare the sh*t out of him, just tell him that the following packages are
going to have to be backlevelled as well.  If there really are things
that can't be undone and redone, that should be a separate highly
visible prompt.

This will require some extended transactional support in urpmi, I would
think, because we'd literally have to overrule rpm about pulling stuff
out from under the feet of other packages if we knew we were going to
put it back.  That would mean that we'd have the responsibility of
ensuring that the transaction either committed fully from our
perspective, or got fully rolled back.

This also means that packagers would have to be aware of packages that
reformat their  application files as the version increa