Re: criterion proposal: upgrading across 2 releases

2016-02-09 Thread Kamil Paral 
> > But I'm not fully convince we really need to be that strict about the
> > choice of words here.
> > 
> > Apart from these, I don't see any further edits needed.
> 
> I'd say we should try to be consistent but not at the cost of making
> the language sound incredibly awkward in context. I'd say go ahead and
> do it as best you can, we can always tweak it later :) Thanks!

OK, I tried to improve the clarity while still keeping the language natural. 
Patches welcome, of course :)

These are my edits:

https://fedoraproject.org/w/index.php?title=Fedora_24_Beta_Release_Criteria=433795=427038

https://fedoraproject.org/w/index.php?title=Fedora_24_Alpha_Release_Criteria=433796=432067

https://fedoraproject.org/w/index.php?title=QA%3ASOP_blocker_bug_process=433797=430030
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-02-09 Thread Kamil Paral 
> On Mon, Feb 8, 2016 at 8:38 AM, Kamil Paral  wrote:
> 
> > Yay for overloaded terms. What we can do is to use "current stable
> > release", "previous stable release" and "Branched release". Do you think
> > that makes sense or do you see a better way?
> 
> Tricky. For ~2 months each year there are three supported current
> stable releases at the same time.
> 
> rawhide
> next-release (or branched)
> current-release
> previous-release
> expiring-release
> expired
> 
> "Branched" is good because it ties the release to the branching
> process, which is an existing familiar term. I don't like combining
> "branched" with the word release though, because this isn't a release
> yet. Whereas 'next-release' suggests it's not yet a release but will
> be. Another plus for next-release is that before branch, it's also
> rawhide, whereas branched(-release) doesn't exist until branch
> happens.

Thanks for the ideas, it helped me to think about this. I didn't use it in my 
current edit, but I quite like "next release".

> 
> previous and expiring could just be referred to as previous.
> 
> stable vs release
> Does using both help? Or is this just wordy? I'm not thinking how the
> combination helps. Is there an unstable release? Are there previous
> unstable releases? Ostensibly Fedora releases stable software, so I
> think stable release is redundant. I'd pick one: i.e. current-stable
> or current-release.

In this case I think it's better to use "current stable release", because we 
also have development releases, and "current release" does not make it clear 
which one it is.

> 
> This way it's possible refer to supported releases as just "release"
> or as *-release rather than "one or more previous stable" or "current
> or previous". Just call them Fedora *-release. It's less wordy.
> 
> Yeah there is a bit of secret decoder ring with this too, but off hand
> I think it's easier to explain/document, remember, and write out than
> long handing everything. In any case, the terms chosen should be
> useful to those who use them the most.
> 
> --
> Chris Murphy
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-02-08 Thread Adam Williamson 
On Mon, 2016-02-08 at 10:38 -0500, Kamil Paral wrote:
> > > 
> > I think the only problem with that is that in the criteria pages I
> > tend
> > to use 'current release' to refer to the release under test. You
> > might
> > need to search through the criteria pages for other occurrences of
> > terms like "current" and "previous" and reconcile those, too -
> > basically come up with some consistent terms and make sure they're
> > all
> > used consistently throughout all three pages.
> 
> Yay for overloaded terms. What we can do is to use "current stable
> release", "previous stable release" and "Branched release". Do you
> think that makes sense or do you see a better way?

I might suggest "the release being tested" instead of "Branched
release", but of course it kinda depends on context.

> But I'm not fully convince we really need to be that strict about the
> choice of words here.
> 
> Apart from these, I don't see any further edits needed.

I'd say we should try to be consistent but not at the cost of making
the language sound incredibly awkward in context. I'd say go ahead and
do it as best you can, we can always tweak it later :) Thanks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-02-08 Thread Kamil Paral 
> > They use our updated terminology of "current" and "previous" Fedora
> > release. To keep things consistent, I propose to change the current
> > criterion [1]:
> > "For each one of the release-blocking package sets, it must be possible to
> > successfully complete an upgrade from a fully updated installation of the
> > previous stable Fedora release with that package set installed. "
> > into
> > "For each one of the release-blocking package sets, it must be possible to
> > successfully complete an upgrade from a fully updated installation of the
> > current and previous stable Fedora release with that package set
> > installed. "
> > 
> > References section would get updated with a link to this thread. The
> > criterion itself does not mention that by upgrade we mean a direct upgrade
> > (not one by one release). If you think it's not clear enough, I can add
> > some wording to the criterion (phrasing suggestions welcome) or add
> > another description section below it.
> > 
> > [1]
> > https://fedoraproject.org/wiki/Fedora_24_Beta_Release_Criteria#Upgrade_requirements
> 
> I think the only problem with that is that in the criteria pages I tend
> to use 'current release' to refer to the release under test. You might
> need to search through the criteria pages for other occurrences of
> terms like "current" and "previous" and reconcile those, too -
> basically come up with some consistent terms and make sure they're all
> used consistently throughout all three pages.

Yay for overloaded terms. What we can do is to use "current stable release", 
"previous stable release" and "Branched release". Do you think that makes sense 
or do you see a better way?

I went through the wiki pages and this would have to change:

https://fedoraproject.org/wiki/Fedora_24_Alpha_Release_Criteria#Desktop_background
Change
"The default desktop background must be different from that of the two previous 
stable releases. "
to
"The default desktop background must be different from the current stable and 
previous stable release. "
(or maybe the original sentence is clear enough?)

https://fedoraproject.org/wiki/Fedora_24_Beta_Release_Criteria#Guest_on_previous_release
Change
" Guest on previous release
The release must install and boot successfully as a virtual guest in a 
situation where the virtual host is running the previous stable Fedora release. 
"
to
" Guest on current stable release
The release must install and boot successfully as a virtual guest in a 
situation where the virtual host is running the current stable Fedora release. "

https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process
This is tricky. We agreed on "AcceptedPreviousRelease" flag name. I wouldn't 
change that. There's also:
"AcceptedPreviousRelease is used for cases where the fix must appear as an 
update for one or more previous stable releases. "
which can be
"AcceptedPreviousRelease is used for cases where the fix must appear as an 
update for the current or previous stable release. "
And
"this will usually need to be fixed in the previous stable releases, not the 
new release"
which can be
"this will usually need to be fixed in the current or previous stable release, 
not the new release"
But I'm not fully convince we really need to be that strict about the choice of 
words here.

Apart from these, I don't see any further edits needed.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-02-08 Thread Chris Murphy 
On Mon, Feb 8, 2016 at 8:38 AM, Kamil Paral  wrote:

> Yay for overloaded terms. What we can do is to use "current stable release", 
> "previous stable release" and "Branched release". Do you think that makes 
> sense or do you see a better way?

Tricky. For ~2 months each year there are three supported current
stable releases at the same time.

rawhide
next-release (or branched)
current-release
previous-release
expiring-release
expired

"Branched" is good because it ties the release to the branching
process, which is an existing familiar term. I don't like combining
"branched" with the word release though, because this isn't a release
yet. Whereas 'next-release' suggests it's not yet a release but will
be. Another plus for next-release is that before branch, it's also
rawhide, whereas branched(-release) doesn't exist until branch
happens.

previous and expiring could just be referred to as previous.

stable vs release
Does using both help? Or is this just wordy? I'm not thinking how the
combination helps. Is there an unstable release? Are there previous
unstable releases? Ostensibly Fedora releases stable software, so I
think stable release is redundant. I'd pick one: i.e. current-stable
or current-release.

This way it's possible refer to supported releases as just "release"
or as *-release rather than "one or more previous stable" or "current
or previous". Just call them Fedora *-release. It's less wordy.

Yeah there is a bit of secret decoder ring with this too, but off hand
I think it's easier to explain/document, remember, and write out than
long handing everything. In any case, the terms chosen should be
useful to those who use them the most.

-- 
Chris Murphy
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-01-25 Thread Kamil Paral 
> > Adding that text to the packaging guidelines seems like a sensible
> > idea, yep. You'd have to propose it in an FPC ticket, that's the
> > procedure for packaging guideline changes IIRC.
> 
> After discussing this on the packaging mailing list, it seems we don't need
> any modifications of the packaging guidelines:
> https://lists.fedoraproject.org/archives/list/packaging%40lists.fedoraproject.org/thread/TAJRGAWPDKBKXUM2M5Y4MG44TEKCYC7J/
> 
> After further consideration, I've also created a FESCo ticket to officially
> approve this:
> https://fedorahosted.org/fesco/ticket/1534

The ticket is not yet updated, but the request was approved:
https://meetbot.fedoraproject.org/teams/fesco/fesco.2016-01-22-17.02.log.html

The new test cases are already in place, we just need to change milestone:
https://fedoraproject.org/wiki/Template:Installation_test_matrix#Upgrade

They use our updated terminology of "current" and "previous" Fedora release. To 
keep things consistent, I propose to change the current criterion [1]:
"For each one of the release-blocking package sets, it must be possible to 
successfully complete an upgrade from a fully updated installation of the 
previous stable Fedora release with that package set installed. "
into
"For each one of the release-blocking package sets, it must be possible to 
successfully complete an upgrade from a fully updated installation of the 
current and previous stable Fedora release with that package set installed. "

References section would get updated with a link to this thread. The criterion 
itself does not mention that by upgrade we mean a direct upgrade (not one by 
one release). If you think it's not clear enough, I can add some wording to the 
criterion (phrasing suggestions welcome) or add another description section 
below it.

[1] 
https://fedoraproject.org/wiki/Fedora_24_Beta_Release_Criteria#Upgrade_requirements
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-01-25 Thread Adam Williamson 
On Mon, 2016-01-25 at 09:18 -0500, Kamil Paral wrote:
> > > Adding that text to the packaging guidelines seems like a sensible
> > > idea, yep. You'd have to propose it in an FPC ticket, that's the
> > > procedure for packaging guideline changes IIRC.
> > 
> > After discussing this on the packaging mailing list, it seems we don't need
> > any modifications of the packaging guidelines:
> > https://lists.fedoraproject.org/archives/list/packaging%40lists.fedoraproject.org/thread/TAJRGAWPDKBKXUM2M5Y4MG44TEKCYC7J/
> > 
> > After further consideration, I've also created a FESCo ticket to officially
> > approve this:
> > https://fedorahosted.org/fesco/ticket/1534
> 
> The ticket is not yet updated, but the request was approved:
> https://meetbot.fedoraproject.org/teams/fesco/fesco.2016-01-22-17.02.log.html
> 
> The new test cases are already in place, we just need to change milestone:
> https://fedoraproject.org/wiki/Template:Installation_test_matrix#Upgrade
> 
> They use our updated terminology of "current" and "previous" Fedora release. 
> To keep things consistent, I propose to change the current criterion [1]:
> "For each one of the release-blocking package sets, it must be possible to 
> successfully complete an upgrade from a fully updated installation of the 
> previous stable Fedora release with that package set installed. "
> into
> "For each one of the release-blocking package sets, it must be possible to 
> successfully complete an upgrade from a fully updated installation of the 
> current and previous stable Fedora release with that package set installed. "
> 
> References section would get updated with a link to this thread. The 
> criterion itself does not mention that by upgrade we mean a direct upgrade 
> (not one by one release). If you think it's not clear enough, I can add some 
> wording to the criterion (phrasing suggestions welcome) or add another 
> description section below it.
> 
> [1] 
> https://fedoraproject.org/wiki/Fedora_24_Beta_Release_Criteria#Upgrade_requirements

I think the only problem with that is that in the criteria pages I tend
to use 'current release' to refer to the release under test. You might
need to search through the criteria pages for other occurrences of
terms like "current" and "previous" and reconcile those, too -
basically come up with some consistent terms and make sure they're all
used consistently throughout all three pages.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2016-01-18 Thread Kamil Paral 
> Adding that text to the packaging guidelines seems like a sensible
> idea, yep. You'd have to propose it in an FPC ticket, that's the
> procedure for packaging guideline changes IIRC.

After discussing this on the packaging mailing list, it seems we don't need any 
modifications of the packaging guidelines:
https://lists.fedoraproject.org/archives/list/packaging%40lists.fedoraproject.org/thread/TAJRGAWPDKBKXUM2M5Y4MG44TEKCYC7J/

After further consideration, I've also created a FESCo ticket to officially 
approve this:
https://fedorahosted.org/fesco/ticket/1534
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-17 Thread Kamil Paral 
> We have +1 from QA, +1 from dnf-plugin-system-upgrade maintainer, and +1 from
> gnome-software maintainers (at least that's how I chose to interpret it).
> There were not many responses from general audience (I hoped for some
> package maintainers feedback). From my POV, we should just do it, or have it
> blessed by FESCo if we want to be extra safe and correct. I would do the
> former.

I forgot two things. There's this quote from Kalev that's worth mentioning:

> With my packager hat on, it would be great if we could get this in the
> packaging guidelines as well, so that there's a canonical source that
> says that obsoletes/conflicts etc must be preserved to support upgrades
> across 2 releases. And also maybe make some noise in devel-announce and
> in the fedora magazine so that packagers are aware that this is
> something everybody needs to support.

What do you think about pushing this into packaging guidelines?

Second, I'd like to highlight that gnome-software is not going to use 
dnf-plugin-system-upgrade, but libhif instead. So we will need to test both 
approaches (that's not specific to skip-release upgrades, but the combinations 
multiply). So this is going to need more resources in OpenQA, and possibly some 
more human resources when debugging issues. I'm not too happy about it, after 
all the effort we put into dnf-plugin-system-upgrade. Still, I think that 
supporting skip-release upgrades doesn't add that much overhead, and it's worth 
it).
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-17 Thread Kamil Paral 
> > So, what's the status of this proposal? I'd say we should go ahead and
> > do it or not before the New Year, ahead of the F24 Alpha ramp-up. Does
> > anyone have further +-1s?

We have +1 from QA, +1 from dnf-plugin-system-upgrade maintainer, and +1 from 
gnome-software maintainers (at least that's how I chose to interpret it). There 
were not many responses from general audience (I hoped for some package 
maintainers feedback). From my POV, we should just do it, or have it blessed by 
FESCo if we want to be extra safe and correct. I would do the former.

> 
> So I can't find this thread, including this post, in the new list web
> UI doing a search for the subject. I did send a "oh by the way" email
> suggested to ping Richard Hughes since the policy likely affects
> whatever is happening with GNOME Software to eventually do the major
> version upgrades. But I can't find that email either.

Here's Kalev's reply:
https://lists.fedoraproject.org/archives/list/devel%40lists.fedoraproject.org/message/OJ6BWZ4KHQ7P7RR7UPCZ4TCOIXCJMZ6E/

Richard's answer doesn't seem to be indexed by hyperkitty, but it was:
"Well, we're not calling into DNF for several technical reasons, but we're 
doing basically the same thing with libhif to what dnf does. I don't see any 
technical reason why we can't do n+2 upgrades, it just makes it harder to test 
all the permutations."
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-17 Thread Adam Williamson 
On Thu, 2015-12-17 at 05:16 -0500, Kamil Paral wrote:
> > We have +1 from QA, +1 from dnf-plugin-system-upgrade maintainer, and +1 
> > from
> > gnome-software maintainers (at least that's how I chose to interpret it).
> > There were not many responses from general audience (I hoped for some
> > package maintainers feedback). From my POV, we should just do it, or have it
> > blessed by FESCo if we want to be extra safe and correct. I would do the
> > former.
> 
> I forgot two things. There's this quote from Kalev that's worth mentioning:
> 
> > With my packager hat on, it would be great if we could get this in the
> > packaging guidelines as well, so that there's a canonical source that
> > says that obsoletes/conflicts etc must be preserved to support upgrades
> > across 2 releases. And also maybe make some noise in devel-announce and
> > in the fedora magazine so that packagers are aware that this is
> > something everybody needs to support.
> 
> What do you think about pushing this into packaging guidelines?
> 
> Second, I'd like to highlight that gnome-software is not going to use
> dnf-plugin-system-upgrade, but libhif instead. So we will need to
> test both approaches (that's not specific to skip-release upgrades,
> but the combinations multiply). So this is going to need more
> resources in OpenQA, and possibly some more human resources when
> debugging issues. I'm not too happy about it, after all the effort we
> put into dnf-plugin-system-upgrade. Still, I think that supporting
> skip-release upgrades doesn't add that much overhead, and it's worth
> it).

I suppose what we could do is test upgrades of some package sets with
DNF and upgrades of others with gnome-software - I guess minimal and
maybe Server with DNF, Workstation with gnome-software.

Adding that text to the packaging guidelines seems like a sensible
idea, yep. You'd have to propose it in an FPC ticket, that's the
procedure for packaging guideline changes IIRC.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-16 Thread Adam Williamson 
On Wed, 2015-12-16 at 10:44 -0700, Chris Murphy wrote:
> On Wed, Dec 16, 2015 at 10:29 AM, Adam Williamson
>  wrote:
> > On Mon, 2015-11-23 at 10:52 -0500, Kamil Paral wrote:
> > 
> > > I feel that for something as important as system upgrade, we should
> > > provide a better level of quality and assurance for upgrading across
> > > 2 releases. Currently we have no criterion and testing it is just an
> > > afterthought, not even tracked anywhere. I'd like to amend the
> > > existing criterion to include N-2 release as well, i.e.:
> > 
> > So, what's the status of this proposal? I'd say we should go ahead and
> > do it or not before the New Year, ahead of the F24 Alpha ramp-up. Does
> > anyone have further +-1s?
> 
> So I can't find this thread, including this post, in the new list web
> UI doing a search for the subject. I did send a "oh by the way" email
> suggested to ping Richard Hughes since the policy likely affects
> whatever is happening with GNOME Software to eventually do the major
> version upgrades. But I can't find that email either.

Last I checked, for some reason, the test@ archives from before the
mailman3 switch hadn't been imported to the new web UI, so you would
only see mails from after the switch there. Possibly that's why. The
old archive is still around - 
https://lists.fedoraproject.org/pipermail/test/ - with all the emails
up to the switchover.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-16 Thread Adam Williamson 
On Wed, 2015-12-16 at 10:54 -0700, Chris Murphy wrote:
> 
> Found mine, but not yours from today.
> https://lists.fedoraproject.org/archives/list/test%40lists.fedoraproject.org/message/EIFYI5WQ7NTPLXISRE47CNZQJZVI734X/

And yeah, the other thing I noticed with the new shiny is that mails
take a bit longer to show up in the web archive. With the old mailman
it was virtually instantaneous, it seems like it can take 15-30 mins
with the new one.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-16 Thread Chris Murphy 
On Wed, Dec 16, 2015 at 10:29 AM, Adam Williamson
 wrote:
> On Mon, 2015-11-23 at 10:52 -0500, Kamil Paral wrote:
>
>> I feel that for something as important as system upgrade, we should
>> provide a better level of quality and assurance for upgrading across
>> 2 releases. Currently we have no criterion and testing it is just an
>> afterthought, not even tracked anywhere. I'd like to amend the
>> existing criterion to include N-2 release as well, i.e.:
>
> So, what's the status of this proposal? I'd say we should go ahead and
> do it or not before the New Year, ahead of the F24 Alpha ramp-up. Does
> anyone have further +-1s?

So I can't find this thread, including this post, in the new list web
UI doing a search for the subject. I did send a "oh by the way" email
suggested to ping Richard Hughes since the policy likely affects
whatever is happening with GNOME Software to eventually do the major
version upgrades. But I can't find that email either.


-- 
Chris Murphy
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-16 Thread Adam Williamson 
On Mon, 2015-11-23 at 10:52 -0500, Kamil Paral wrote:

> I feel that for something as important as system upgrade, we should
> provide a better level of quality and assurance for upgrading across
> 2 releases. Currently we have no criterion and testing it is just an
> afterthought, not even tracked anywhere. I'd like to amend the
> existing criterion to include N-2 release as well, i.e.:

So, what's the status of this proposal? I'd say we should go ahead and
do it or not before the New Year, ahead of the F24 Alpha ramp-up. Does
anyone have further +-1s?

As mentioned, we do now have openQA testing minimal and Workstation
upgrades from F22 and F23 each day (and in fact it looks like it caught
a bug in Workstation upgrades, today...)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-16 Thread Chris Murphy 
On Wed, Dec 16, 2015 at 10:44 AM, Chris Murphy  wrote:
> On Wed, Dec 16, 2015 at 10:29 AM, Adam Williamson
>  wrote:
>> On Mon, 2015-11-23 at 10:52 -0500, Kamil Paral wrote:
>>
>>> I feel that for something as important as system upgrade, we should
>>> provide a better level of quality and assurance for upgrading across
>>> 2 releases. Currently we have no criterion and testing it is just an
>>> afterthought, not even tracked anywhere. I'd like to amend the
>>> existing criterion to include N-2 release as well, i.e.:
>>
>> So, what's the status of this proposal? I'd say we should go ahead and
>> do it or not before the New Year, ahead of the F24 Alpha ramp-up. Does
>> anyone have further +-1s?
>
> So I can't find this thread, including this post, in the new list web
> UI doing a search for the subject. I did send a "oh by the way" email
> suggested to ping Richard Hughes since the policy likely affects
> whatever is happening with GNOME Software to eventually do the major
> version upgrades. But I can't find that email either.

Found mine, but not yours from today.
https://lists.fedoraproject.org/archives/list/test%40lists.fedoraproject.org/message/EIFYI5WQ7NTPLXISRE47CNZQJZVI734X/

Anyway, I'm a +1.

I am a bit concerned about how GRUB gets stale on BIOS computers with
upgrades rather than new installations, since grub2-install isn't
called with any of the upgrade methods we've had.

http://hmarco.org/bugs/CVE-2015-8370-Grub2-authentication-bypass.html

That is fixed today in grub2-2.02-0.25.fc23. On UEFI, grubx64.efi is
replaced so it's fixed there just by updating the package. But on BIOS
it'll require a manual grub2-install.




-- 
Chris Murphy
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-07 Thread Tim Flink 
On Mon, 23 Nov 2015 13:26:54 -0500 (EST)
Kamil Paral  wrote:

> > I feel that for something as important as system upgrade, we should
> > provide a better level of quality and assurance for upgrading
> > across 2 releases. Currently we have no criterion and testing it is
> > just an afterthought, not even tracked anywhere. I'd like to amend
> > the existing criterion to include N-2 release as well, i.e.:
> > 
> > "For each one of the release-blocking package sets, it must be
> > possible to successfully complete an upgrade from a fully updated
> > installation of any of the two previous stable Fedora releases with
> > that package set installed." (language corrections very welcome)
> 
> During QA meeting, people were generally in favor of this idea. I was
> asked to check with Will Woods, system-upgrade maintainer. I did
> that, and he has no objections. He says it's no extra work for him
> and that the most work is needed from package maintainers when
> designing dependencies, scriptlets, etc. This is something to
> consider as well, but I think we implicitly want all packages to be
> upgradable forward (even when skipping a release). And I don't
> remember any issues with this lately.
> 
> Does someone have some additional concerns about this?

I'm +1 on N-2 upgrades blocking N's release. It'd be nice to have more
automated upgrade tests, maybe with a few different configs but that's
more into the details.

Tim


pgptbNa00DQPV.pgp
Description: OpenPGP digital signature
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-12-07 Thread Adam Williamson 
On Mon, 2015-12-07 at 09:46 -0700, Tim Flink wrote:
> On Mon, 23 Nov 2015 13:26:54 -0500 (EST)
> Kamil Paral  wrote:
> 
> > > I feel that for something as important as system upgrade, we should
> > > provide a better level of quality and assurance for upgrading
> > > across 2 releases. Currently we have no criterion and testing it is
> > > just an afterthought, not even tracked anywhere. I'd like to amend
> > > the existing criterion to include N-2 release as well, i.e.:
> > > 
> > > "For each one of the release-blocking package sets, it must be
> > > possible to successfully complete an upgrade from a fully updated
> > > installation of any of the two previous stable Fedora releases with
> > > that package set installed." (language corrections very welcome)
> > 
> > During QA meeting, people were generally in favor of this idea. I was
> > asked to check with Will Woods, system-upgrade maintainer. I did
> > that, and he has no objections. He says it's no extra work for him
> > and that the most work is needed from package maintainers when
> > designing dependencies, scriptlets, etc. This is something to
> > consider as well, but I think we implicitly want all packages to be
> > upgradable forward (even when skipping a release). And I don't
> > remember any issues with this lately.
> > 
> > Does someone have some additional concerns about this?
> 
> I'm +1 on N-2 upgrades blocking N's release. It'd be nice to have more
> automated upgrade tests, maybe with a few different configs but that's
> more into the details.

As of ~5 minutes ago, openQA tests both N-1 and N-2 upgrades; it does
'minimal' and 'desktop' upgrades on x86_64, and just 'desktop' on i686.
Note that up until now, openQA's F24 upgrade tests have actually been
N-2 upgrades, because we have to manually rebuild the base install
images and edit the openQA test config to update the upgrade tests, and
we hadn't done that yet (so it was still using F22 base images for the
upgrade tests). I've now bumped the original upgrade tests to F23 and
added new 'upgrade_2' tests for N-2, which are currently set to F22 of
course.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-25 Thread Adam Williamson 
On Wed, 2015-11-25 at 09:46 -0500, Richard Ryniker wrote:
> System-upgrade across two releases raises an interesting End-of-Life
> policy issue.  If system-upgrade from N-2 to N is so important it will
> block release of N until it works, how do we explain why it is no
> longer important after four weeks when N-2 reaches End of Life?

It's not like dnf-system-upgrade would magically stop working when N-2
reaches EOL, so honestly, overall, I just don't really see the problem
you're describing in this mail. We've had the current release scheme
for years, people are fairly used to it; I don't honestly see that
officially supporting N-2 upgrades changes the overall story much,
we're just giving people another tool to use in the one month
transition period (and of course, in practice, dnf-system-upgrade
should usually continue to work just fine after EOL).
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-25 Thread Adam Williamson 
On Wed, 2015-11-25 at 13:36 -0500, Richard Ryniker wrote:
> My reading of the EOL policy says "If it isn't fixed before EOL, it is
> unlikely to be fixed."  If I encounter a problem with release-skipping
> system-upgrade, too bad.  There is probably no time to fix it before EOL.

That's not really accurate. A month is a pretty long time in Fedora
development, certainly long enough to diagnose and fix typical upgrade
issues.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-25 Thread Adam Williamson 
On Wed, 2015-11-25 at 16:51 -0500, Richard Ryniker wrote:
> > A month is a pretty long time in Fedora development
> 
> True, but a month is only available if the problem is reported on day 1.
> If it takes a week or two for a user to report a problem, that interval
> lessens the remaining time to EOL.

On the other other hand, you can test upgrades at any time, with dnf-
system-upgrade. We're already testing 22-24 upgrades daily in openQA at
present.

> On the other hand, there is no prohibition against a fix after EOL.

There is, in fact; you can't push updates for EOL releases, it's simply
disabled by policy.

> We also do not know there will be serious problems.  We know it is
> impossible to test more than a miniscule fraction of possible upgrade
> cases.  That does not mean there will be lots of bugs, it means we have
> little confidence the test plan has found most bugs.  We might say there
> is no reason to believe experience with the current release will be any
> worse than the last.

This is basically what I'm saying: we can't actually make any
guarantees about *most* upgrade experiences, and N-2 upgrades aren't
really some special case which is massively more likely to go wrong
than any other. Will we have people who have issues doing N-2 upgrades?
Probably, sure. Do we have people who have issues doing N-1 upgrades?
yep! This isn't something new. We *do*, I think, mention in all the
relevant documentation that upgrades are complex operations and success
is never guaranteed, that's not what 'supported' means, especially in
this context.

> As far as I know, this is the first time "dnf system-upgrade" has been
> generally available for a release-skip upgrade.  It is way too early to
> have any historical perspective.

F23 is the first release where the dnf-system-upgrade mechanism has
been available, yes. But it's really not hugely different from fedup -
it's just an even lighter implementation of the same basic idea (just
boot to a minimal environment and run a dnf transaction). The
experience of fedup should be a reasonable indicator of future dnf-
system-upgrade experiences.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-25 Thread Richard Ryniker 
>A month is a pretty long time in Fedora development

True, but a month is only available if the problem is reported on day 1.
If it takes a week or two for a user to report a problem, that interval
lessens the remaining time to EOL.

On the other hand, there is no prohibition against a fix after EOL.

We also do not know there will be serious problems.  We know it is
impossible to test more than a miniscule fraction of possible upgrade
cases.  That does not mean there will be lots of bugs, it means we have
little confidence the test plan has found most bugs.  We might say there
is no reason to believe experience with the current release will be any
worse than the last.

As far as I know, this is the first time "dnf system-upgrade" has been
generally available for a release-skip upgrade.  It is way too early to
have any historical perspective.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-25 Thread Richard Ryniker 
>It's not like dnf-system-upgrade would magically stop working when N-2
>reaches EOL, so honestly, overall, I just don't really see the problem
>you're describing

Like most of Fedora, dnf-system-upgrade gets limited testing before
release.  When N is released, a large number of users who did not install
N-1 will think it is time to upgrade before their current N-2 reaches
End-of-Life in four weeks.  These users will try system-upgrade from a
much larger set of initial states than could be tested before release.
This is the time the greatest number of (N-2 to N) problems will be found.

It is also the time when problems in the brand-new release N are most
likely, as users of N-1 eagerly try N.

With an abundance of bug reports in the just-released N, and some new
reports of problems with N-2 that will be closed by EOL in a handful of
days, where do you think maintenance should focus?

I admit this view is based on what is probable, not on actual fact.  My
point is not do this or avoid that, I want the Fedora user to have an
accurate understanding of upgrade choices.

My reading of the EOL policy says "If it isn't fixed before EOL, it is
unlikely to be fixed."  If I encounter a problem with release-skipping
system-upgrade, too bad.  There is probably no time to fix it before EOL.

In effect, release-skipping system-upgrade is not supported.  Either I
upgrade every release (when system-upgrade is supported and bug fixes are
likely), I plan to do a new install, or I hope to be lucky with EOL code.

That is not a bad set of choices.  Bad is a user in trouble because he
reasonably thought they were different.  "Release-skipping system-upgrade
is a release-blocking requirement" sounds likely to obscure the support
risks that attend its use.

Fedora is certainly better for a robust system-upgrade facility.  No
point is filing bugs now for my 21-23 failure, though.  21 is EOL, and
the failure did not occur with 22.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-25 Thread Richard Ryniker 
System-upgrade across two releases raises an interesting End-of-Life
policy issue.  If system-upgrade from N-2 to N is so important it will
block release of N until it works, how do we explain why it is no
longer important after four weeks when N-2 reaches End of Life?

Four weeks is little time to allow users who chose not to upgrade to N-1
to move from N-2 to N.  Do we say "After four weeks, the only supported
mechanismm is new installation of N"?  That is clear and simple,
consistent with the EOL schedule, but denies system-upgrade is such a
critical facility.

Should system-upgrade be formally supported for three releases (18
months?)  That would tell users they may count on it for their upgrade
plan, but it is a significant change that could involve FESCo.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Richard Ryniker 
On Tue, 24 Nov 2015 10:46:55 -0800, Adam Williamson wrote:

> We have never really gone to any lengths to test or support N-1 upgrades
> with 3rd party repositories or non-repo software either.  That's a
> different question.

>From a user's perspective, the value of system-upgrade depends on its
ability to move his system "as configured" to a new release.  All of his
personalization, including any non-fedora software, remains in place.

You are certainly right to except non-Fedora software from QA tests.  The
problem I perceive is the vast number of possible combinations of
packages from just the Fedora repositories.  There may be no need to test
exhaustively, but any heuristic is likely to miss problems from time to
time.  It is likely only enough cases can be tested to distinguish "some
system-upgrades work" from "system-upgrade is often impossible."

This is valuable to know, and I have no quibble with a QA criterion that
says "system-upgrade is never possible" will block a release.  I suspect
there is no precise, useful definition that will tell a user whether his
particular system will succeed with system-upgrade.  All he can do is try
and hope for the best.  There might be documentation that says "these
circumstances are likely to prevent successful system-upgrade" - the
"common bugs" approach.

With any general claim "You can use system-upgrade to advance to the next
(or next+1) release" there will be triage issues (how many users will
experience problem 1; how many will encounter problem 2?)  How many
susceptible users must we estimate in order to block a release?  I fear
software developers will rightly claim they are busy with new features or
current bugs and they cannot spend time to investigate compatibility
problems from system-upgrade that simply do not appear when their sofware
is newly installed.

I do not want a Fedora user to understand "You can use system-upgrade to
migrate to a newer release" has the same confidence as "The new Fedora
meets all release criteria."

> Why does the situation inevitably 'get worse' for N-2 upgrade?

There are more possible initial conditions.  Even a very sparse test plan
requires more effort.  Analysis of "How important is this case?" may be
more difficult, and there are more cases.  Instead of problems from six
months of software evolution, there will be problems from 12 months, and
the increase in number of problems over time is likely greater than
linear.

I certainly do not want to abandon system-upgrade.  I want users to
understand we cannot test it well, probably cannot fix many failures,
and, when it fails, we reccommend they report the problem then undertake
a new installation of Fedora, configure it, and reload applications to
meet their requirements.

"System-upgrade must work before release" is too vague ("work" is too
difficult to define and test), too severe (important new function could
be delayed for reasons no one is willing to fix), and will tell users who
experience problems that Fedora is lower in quality than they hoped.

If users are told up front that system-upgrade is useful but cannot be
tested or guaranteed for all cases, their expectations will be more
realistic, therefore satisfaction with Fedora will be greater.

"dnf system-upgrade" is rather new.  Maybe the Fedora organization should
observe user experience for another release or two before it decides this
is a critical, and therefore release-blocking, part of Fedora.  I
understand enthusiasm for the new, offline upgrade mechanism.  More
history will yield a better understanding of what problems can occur, and
what costs must be borne to fix them.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Adam Williamson 
On Tue, 2015-11-24 at 21:57 -0500, Richard Ryniker wrote:
> On Tue, 24 Nov 2015 10:46:55 -0800, Adam Williamson wrote:
> 
> > We have never really gone to any lengths to test or support N-1 upgrades
> > with 3rd party repositories or non-repo software either.  That's a
> > different question.
> 
> From a user's perspective, the value of system-upgrade depends on its
> ability to move his system "as configured" to a new release.  All of his
> personalization, including any non-fedora software, remains in place.
> 
> You are certainly right to except non-Fedora software from QA tests.  The
> problem I perceive is the vast number of possible combinations of
> packages from just the Fedora repositories.  There may be no need to test
> exhaustively, but any heuristic is likely to miss problems from time to
> time.  It is likely only enough cases can be tested to distinguish "some
> system-upgrades work" from "system-upgrade is often impossible."
> 
> This is valuable to know, and I have no quibble with a QA criterion that
> says "system-upgrade is never possible" will block a release.  I suspect
> there is no precise, useful definition that will tell a user whether his
> particular system will succeed with system-upgrade.

Yes, this is basically the situation. We can only do so much by
literally testing actual upgrades, by hand or automated. Things like
the daily dependency checks and the taskotron depcheck test can help
more. But ultimately, there are so many permutations that we can never
really offer a guaranteed smooth upgrade path (and in fact, AFAIK, no
OS vendor - even those with far more resources than us - does this).

> "dnf system-upgrade" is rather new.  Maybe the Fedora organization should
> observe user experience for another release or two before it decides this
> is a critical, and therefore release-blocking, part of Fedora.  I
> understand enthusiasm for the new, offline upgrade mechanism.  More
> history will yield a better understanding of what problems can occur, and
> what costs must be borne to fix them.

dnf-system-upgrade itself is new, but Fedora has always had a
theoretically 'official supported' upgrade mechanism; it was anaconda,
then fedup, now it's dnf-system-upgrade. dnf-system-upgrade was
introduced specifically with the understanding that it replaced fedup
at the same level of support.

I don't think this is enthusiasm for dnf-system-upgrade per se, just a
kind of recognition of a gap in coverage that's been around for a
while. We could probably have plausibly supported N+2 upgrades with
fedup; certainly in practice we put effort into fixing them.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Richard Ryniker 
On Tue, 24 Nov 2015 07:52:47 -0500, Stephen Gallagher wrote:

> I've upgraded several family members directly from Fedora 21 to Fedora 23
> in the last week with no issues whatsoever (of course, I also curate
> their repository selection, so they don't end up with incompatible
> packages).

Aye, there's the rub.  It is not just update from an "original"
installation of N-2 to N, it is update from N-2 plus whatever additional
repository packages were installed plus additional non-repository
software and user configuration that you want to upgrade.  The starting
condition is quite variable, and may include old versions of applications
because these are familiar to the user or incompatible with more
up-to-date software.

When system-upgrade fails (if you are lucky, this will be obvious; if
unlucky, it may not be recognized until after the event), the only way
back to the starting condition is a full system restore.  Many users will
back up their own data, and, if upgrade ends badly, plan to recover with
a new installation (of N, N-1, or N-2) and necessary software, followed
by restoration of their personal data.  This is the most reliable upgrade
strategy: no exposure to problems with old stuff, and it defines how to
get from a well-known initial state to the user's working environment.

In this case it may be impossible to fix an upgrade problem, because
there is no way to reconstruct the original state and verify successful
upgrade is possible.

Of course, these considerations also apply to the single-release upgrade.
The situation just gets worse the farther back one goes.  And sometimes
there simply is no upgrade path: the new software is just incompatible
with the old.  To use the new system, you must adapt to it.  (Remember
the agony voiced by some GNOME2 users when GNOME3 came along?)

Unlike new installation with its well-defined result, system-upgrade
(with dependencies on user configuration and possibly incompatible pieces
of software) necessarily has some vague result.  It is undeniably
valuable and extremely convenient, but unavoidably a second-class
facility.  We may insist system-upgrade of a newly installed N-1 to N
works, and further insist system-upgrade of a newly installed N-2 to N
works, but this is deceptive.  Once a user configures and uses his
system, system-upgrade is unpredictable.

To add my own to Mr. Gallagher's anecdote, my single attempt at "dnf
system-upgrade" from 21 to 23 failed.  After upgrade, my userid was not
displayed on the login screen.  Login in text mode was possible.  After
an hour or two tinkering with gdm configuration, I gave up, did a new
install of 23, and configured it without difficulty.  Several times I
have used "dnf system-upgrade" from 22 to 23 without problems.

Conclusion:  release-skipping upgrade is more difficult than
single-release upgrade (and probably not worth the effort to fix.)

If anyone would like to investigate the problem I experienced, I am
willing to install a new 21 system and see if I can reproduce the
error.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Adam Williamson 
On Tue, 2015-11-24 at 13:28 -0500, Richard Ryniker wrote:
> On Tue, 24 Nov 2015 07:52:47 -0500, Stephen Gallagher wrote:
> 
> > I've upgraded several family members directly from Fedora 21 to Fedora 23
> > in the last week with no issues whatsoever (of course, I also curate
> > their repository selection, so they don't end up with incompatible
> > packages).
> 
> Aye, there's the rub.  It is not just update from an "original"
> installation of N-2 to N, it is update from N-2 plus whatever additional
> repository packages were installed plus additional non-repository
> software and user configuration that you want to upgrade. 

Not really. We have never really gone to any lengths to test or support
N-1 upgrades with 3rd party repositories or non-repo software either.
That's a different question.

> Of course, these considerations also apply to the single-release upgrade.
> The situation just gets worse the farther back one goes.

I'm not really sure that logically follows. Why does the situation
inevitably 'get worse' for N-2 upgrade?

Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Adam Williamson 
On Tue, 2015-11-24 at 07:52 -0500, Stephen Gallagher wrote:

> 1) Way back when we were doing Anaconda-based upgrades, I think it
> amounted to a technological limitation (please correct me if my
> information is wrong here).

I think it wasn't exactly technically *impossible*, but it was
significantly more likely that some kind of problem would emerge, with
that much heavier system.

> 2) The QA impact is non-zero when attempting to support two-version
> upgrades, but since this recommendation is coming *from* prominent QA
> team members, I'm assuming that this is a non-issue.

Not exactly a non-issue, but we at least have the ability to automate
this testing quite effectively now.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Kamil Paral 
> I believe a failure to upgrade from N-2 to N should not block the N
> release.  The reason is limited resources, both for tests and for changes
> to fix problems.  These resources are more valuable applied to the N
> release than to something two releases in the past.
> 
> If someone wants to test a release-skipping upgrade, fine.  Report
> problems?  Sure.  If someone wants to fix these problems, OK; if not, the
> policy should be "Upgrade one release at a time.  Release-skipping
> upgrades may work, but are not guaranteed."

That's exactly what I'm trying to improve here. Either make it a reliable 
feature, or warn the users against it very visibly, because this is an 
operation that can easily break their system heavily. We don't currently warn 
the users much, especially the system-upgrade tool doesn't contain any warnings 
like that. If we decide we're not going to support release skipping officially, 
users should always do a conscious and informed decision about this and be 
aware of the risks.

> 
> If "upgrade N-2 -> N-1" works, and "upgrade N-1 -> N" works, then "upgrade
> N-2 -> N" also works, right?  Maybe not, and I think it profligate to
> insist we fix a broken two-release upgrade when two single-release
> upgrades successfully reach the desired target.  Document, do not block.
> 
> I may hold a minority opinion, but this seems like another call for QA to
> do somebody else's job.  Who should decide that release-skip upgrade is a
> Fedora imperative?

That would be FESCo. If more people hold your opinion, we should probably file 
a ticket and ask them. It's definitely a valid opinion. And it would definitely 
be easier for QA. But in this case, I believe the work is worth it, because I 
personally know many people who don't want to upgrade twice a year. And 
offering people a way to upgrade just once a year (easily, i.e. not going 
through consecutive upgrades) is something I consider essential for an 
operating system.
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-24 Thread Stephen Gallagher 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/24/2015 06:48 AM, Kamil Paral wrote:
> That would be FESCo. If more people hold your opinion, we should 
> probably file a ticket and ask them. It's definitely a valid
> opinion. And it would definitely be easier for QA. But in this
> case, I believe the work is worth it, because I personally know
> many people who don't want to upgrade twice a year. And offering
> people a way to upgrade just once a year (easily, i.e. not going
> through consecutive upgrades) is something I consider essential for
> an operating system.


With my FESCo hat on, I'll say this: I'd prefer that Fedora officially
supports the two-release upgrade path. From anecdotal data, a
significant percentage of our non-developer user-base is doing this
already and is comfortable with the existing risks. Improving the
situation can only serve to enhance our standing and reputation.

Historically, the reason we have not done this was twofold:

1) Way back when we were doing Anaconda-based upgrades, I think it
amounted to a technological limitation (please correct me if my
information is wrong here).

2) The QA impact is non-zero when attempting to support two-version
upgrades, but since this recommendation is coming *from* prominent QA
team members, I'm assuming that this is a non-issue.


Also, to add to the anecdotes, I've upgraded several family members
directly from Fedora 21 to Fedora 23 in the last week with no issues
whatsoever (of course, I also curate their repository selection, so
they don't end up with incompatible packages). So I'd argue that we're
already at the point where this is a real functional capability that
we should just ensure doesn't get broken in the future (as opposed to
needing to add new capabilities to enable two-version upgrades).
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iEYEARECAAYFAlZUXZwACgkQeiVVYja6o6Mf6gCgmxG38vX0SqjSt2er9o6XHZRp
oyQAniy4UE8lpcOP4kikoSHaPucR+G1b
=MjGD
-END PGP SIGNATURE-
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-23 Thread Kamil Paral 
> I feel that for something as important as system upgrade, we should provide a
> better level of quality and assurance for upgrading across 2 releases.
> Currently we have no criterion and testing it is just an afterthought, not
> even tracked anywhere. I'd like to amend the existing criterion to include
> N-2 release as well, i.e.:
> 
> "For each one of the release-blocking package sets, it must be possible to
> successfully complete an upgrade from a fully updated installation of any of
> the two previous stable Fedora releases with that package set installed."
> (language corrections very welcome)

During QA meeting, people were generally in favor of this idea. I was asked to 
check with Will Woods, system-upgrade maintainer. I did that, and he has no 
objections. He says it's no extra work for him and that the most work is needed 
from package maintainers when designing dependencies, scriptlets, etc. This is 
something to consider as well, but I think we implicitly want all packages to 
be upgradable forward (even when skipping a release). And I don't remember any 
issues with this lately.

Does someone have some additional concerns about this?

Thanks,
Kamil
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-23 Thread Richard Ryniker 
I believe a failure to upgrade from N-2 to N should not block the N
release.  The reason is limited resources, both for tests and for changes
to fix problems.  These resources are more valuable applied to the N
release than to something two releases in the past.

If someone wants to test a release-skipping upgrade, fine.  Report
problems?  Sure.  If someone wants to fix these problems, OK; if not, the
policy should be "Upgrade one release at a time.  Release-skipping
upgrades may work, but are not guaranteed."

If "upgrade N-2 -> N-1" works, and "upgrade N-1 -> N" works, then "upgrade
N-2 -> N" also works, right?  Maybe not, and I think it profligate to
insist we fix a broken two-release upgrade when two single-release
upgrades successfully reach the desired target.  Document, do not block.

I may hold a minority opinion, but this seems like another call for QA to
do somebody else's job.  Who should decide that release-skip upgrade is a
Fedora imperative?
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: criterion proposal: upgrading across 2 releases

2015-11-23 Thread Adam Williamson 
On Mon, 2015-11-23 at 13:26 -0500, Kamil Paral wrote:
> > I feel that for something as important as system upgrade, we should provide 
> > a
> > better level of quality and assurance for upgrading across 2 releases.
> > Currently we have no criterion and testing it is just an afterthought, not
> > even tracked anywhere. I'd like to amend the existing criterion to include
> > N-2 release as well, i.e.:
> > 
> > "For each one of the release-blocking package sets, it must be possible to
> > successfully complete an upgrade from a fully updated installation of any of
> > the two previous stable Fedora releases with that package set installed."
> > (language corrections very welcome)
> 
> During QA meeting, people were generally in favor of this idea. I was
> asked to check with Will Woods, system-upgrade maintainer. I did
> that, and he has no objections. He says it's no extra work for him
> and that the most work is needed from package maintainers when
> designing dependencies, scriptlets, etc. This is something to
> consider as well, but I think we implicitly want all packages to be
> upgradable forward (even when skipping a release). And I don't
> remember any issues with this lately.
> 
> Does someone have some additional concerns about this?

Nope - as I said in the meeting, if tools folks are OK, so am I. It may
be worth floating on devel@, though, to see if any packagers are
concerned about maintaining upgradability on that level.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org